In [2]:
Collecting pydicom
  Downloading https://files.pythonhosted.org/packages/f4/15/df16546bc59bfca390cf072d473fb2c8acd4231636f64356593a63137e55/pydicom-2.1.2-py3-none-any.whl (1.9MB)
     |████████████████████████████████| 1.9MB 4.7MB/s 
Installing collected packages: pydicom
Successfully installed pydicom-2.1.2
In [ ]:
TensorFlow 1.x selected.
In [ ]:
Requirement already satisfied: keras==2.1.5 in /usr/local/lib/python3.7/dist-packages (2.1.5)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from keras==2.1.5) (3.13)
Requirement already satisfied: scipy>=0.14 in /usr/local/lib/python3.7/dist-packages (from keras==2.1.5) (1.4.1)
Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.7/dist-packages (from keras==2.1.5) (1.15.0)
Requirement already satisfied: numpy>=1.9.1 in /usr/local/lib/python3.7/dist-packages (from keras==2.1.5) (1.19.5)
In [3]:

Above we configure our notebook for googel colab, install/import necessary libraries and versions

In [ ]:

Configuring our train class info csv file as classes

Configuring our training image set info as train

In [ ]:
Out[6]:
patientId x y width height Target
0 0004cfab-14fd-4e49-80ba-63a80b6bddd6 NaN NaN NaN NaN 0
1 00313ee0-9eaa-42f4-b0ab-c148ed3241cd NaN NaN NaN NaN 0
2 00322d4d-1c29-4943-afc9-b6754be640eb NaN NaN NaN NaN 0
3 003d8fa0-6bf1-40ed-b54c-ac657f8495c5 NaN NaN NaN NaN 0
4 00436515-870c-4b36-a041-de91049b9ab4 264.0 152.0 213.0 379.0 1
... ... ... ... ... ... ...
30222 c1ec14ff-f6d7-4b38-b0cb-fe07041cbdc8 185.0 298.0 228.0 379.0 1
30223 c1edf42b-5958-47ff-a1e7-4f23d99583ba NaN NaN NaN NaN 0
30224 c1f6b555-2eb1-4231-98f6-50a963976431 NaN NaN NaN NaN 0
30225 c1f7889a-9ea9-4acb-b64c-b737c929599a 570.0 393.0 261.0 345.0 1
30226 c1f7889a-9ea9-4acb-b64c-b737c929599a 233.0 424.0 201.0 356.0 1

30227 rows × 6 columns

Train has patient id's of patients along with bounding box coordinates of the opacity in the xray

We have a traget column which is binary and tells us if that particular patinet had pneumonia or not

In [ ]:
Out[7]:
patientId class
0 0004cfab-14fd-4e49-80ba-63a80b6bddd6 No Lung Opacity / Not Normal
1 00313ee0-9eaa-42f4-b0ab-c148ed3241cd No Lung Opacity / Not Normal
2 00322d4d-1c29-4943-afc9-b6754be640eb No Lung Opacity / Not Normal
3 003d8fa0-6bf1-40ed-b54c-ac657f8495c5 Normal
4 00436515-870c-4b36-a041-de91049b9ab4 Lung Opacity
... ... ...
30222 c1ec14ff-f6d7-4b38-b0cb-fe07041cbdc8 Lung Opacity
30223 c1edf42b-5958-47ff-a1e7-4f23d99583ba Normal
30224 c1f6b555-2eb1-4231-98f6-50a963976431 Normal
30225 c1f7889a-9ea9-4acb-b64c-b737c929599a Lung Opacity
30226 c1f7889a-9ea9-4acb-b64c-b737c929599a Lung Opacity

30227 rows × 2 columns

Classes also has patient id's along with the class that it belongs to

In [ ]:
Out[8]:
array(['No Lung Opacity / Not Normal', 'Normal', 'Lung Opacity'],
      dtype=object)

Classes here are

1. No Lung Opacity / Not Normal

2. Normal

3. Lung Opacity

Here we get a baisc idea of whether the patient tested positive or negative for pneumonia

In [ ]:
Classes Summary-----> Data Points: 30227, Classes: 2
Train Data Summary-----> Data Points: 30227, features: 6

Our classes dataframe has 30227 rows and 2 columns

Our train dataframe has 30227 rows and 6 columns

In [ ]:

This above function will help us check the missing values and the % number compared to full data

In [ ]:

This function gives us the % of data in each class against the total

In [ ]:

The above function helps us explore and view our images along with the dicom files. Medical images are preserved in peculiar format called DICOM files with extension *.dcm . It has combination of header metadata and underlying raw image array. Python has support for this and the best module to be used is the pydicom module

In [ ]:

Same fucntion but with bounding boxes

In [ ]:
Out[13]:
height width y x Target patientId
Total 20672.000000 20672.000000 20672.000000 20672.000000 0.0 0.0
Percent 68.389188 68.389188 68.389188 68.389188 0.0 0.0

We check missing values in our train dataframe

There are an ample amount in co ordinates since xray images with no Pneumonia needs no bouding boxes

Hence H,W,X,W would be nan for class 0

We shall not change it and let it be, because these will be handy further, since we can easily separte classes by filtering with np.nan

In [ ]:
Out[14]:
class patientId
Total 0.0 0.0
Percent 0.0 0.0

The classes dataframe is free of null values

In [ ]:
/Users/chiraglodaya/opt/anaconda3/lib/python3.8/site-packages/seaborn/_decorators.py:36: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  warnings.warn(
In [ ]:
Feature: class
No Lung Opacity / Not Normal  :   11821 or 39.1%
Lung Opacity                  :   9555 or 31.61%
Normal                        :   8851 or 29.28%

Above we see our target class distribution

No Lung Opacity / Not Normal is ~ 39% which is negative class along with another negative class Normal which is ~30%

Our positive class Lung Opacity is ~31%

In [ ]:
Out[17]:
patientId x y width height Target class
0 0004cfab-14fd-4e49-80ba-63a80b6bddd6 NaN NaN NaN NaN 0 No Lung Opacity / Not Normal
1 00313ee0-9eaa-42f4-b0ab-c148ed3241cd NaN NaN NaN NaN 0 No Lung Opacity / Not Normal
2 00322d4d-1c29-4943-afc9-b6754be640eb NaN NaN NaN NaN 0 No Lung Opacity / Not Normal
3 003d8fa0-6bf1-40ed-b54c-ac657f8495c5 NaN NaN NaN NaN 0 Normal
4 00436515-870c-4b36-a041-de91049b9ab4 264.0 152.0 213.0 379.0 1 Lung Opacity
... ... ... ... ... ... ... ...
37624 c1f6b555-2eb1-4231-98f6-50a963976431 NaN NaN NaN NaN 0 Normal
37625 c1f7889a-9ea9-4acb-b64c-b737c929599a 570.0 393.0 261.0 345.0 1 Lung Opacity
37626 c1f7889a-9ea9-4acb-b64c-b737c929599a 570.0 393.0 261.0 345.0 1 Lung Opacity
37627 c1f7889a-9ea9-4acb-b64c-b737c929599a 233.0 424.0 201.0 356.0 1 Lung Opacity
37628 c1f7889a-9ea9-4acb-b64c-b737c929599a 233.0 424.0 201.0 356.0 1 Lung Opacity

37629 rows × 7 columns

We next merge the datafarmes into a single train dataframe for better usgae

In [ ]:

We now check overall distribution of our classes positive and negative

Positive is almost 69% and negative class is 31%

We get an idea that class imbalance is not a case here

In [ ]:
/Users/chiraglodaya/opt/anaconda3/lib/python3.8/site-packages/seaborn/distributions.py:2551: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).
  warnings.warn(msg, FutureWarning)
/Users/chiraglodaya/opt/anaconda3/lib/python3.8/site-packages/seaborn/distributions.py:2551: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).
  warnings.warn(msg, FutureWarning)
/Users/chiraglodaya/opt/anaconda3/lib/python3.8/site-packages/seaborn/distributions.py:2551: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).
  warnings.warn(msg, FutureWarning)
/Users/chiraglodaya/opt/anaconda3/lib/python3.8/site-packages/seaborn/distributions.py:2551: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).
  warnings.warn(msg, FutureWarning)
<Figure size 432x288 with 0 Axes>

We check the distribution of our x,y,h,w

Y, H, W seem to be normally distributed but X seems worrysome with bimodal disrtibution which may cause problems further

In [ ]:

We next consider 2000 samples of pneumonia data and jot the center of the binding boxes

The red patches are the entire boxes and black points are the center of those boxes

With this we get an very important idea that no specific part of the lung may be prone to opacity

Almost we have every part of lung in 2000 samples has opacity

In [ ]:
Number of images in train set: 26684 
Number of images in test set: 3000

So we check the train and test set, train set excluding repeated images is 26684 and test has 3000 samples

This much data is good to train a Deep learning model from scratch with decent accuracy and more than enough to feed pre trained model

In [ ]:
Unique patientId in  training set:  26684
In [ ]:
Out[23]:
Dataset.file_meta -------------------------------
(0002, 0000) File Meta Information Group Length  UL: 202
(0002, 0001) File Meta Information Version       OB: b'\x00\x01'
(0002, 0002) Media Storage SOP Class UID         UI: Secondary Capture Image Storage
(0002, 0003) Media Storage SOP Instance UID      UI: 1.2.276.0.7230010.3.1.4.8323329.28530.1517874485.775526
(0002, 0010) Transfer Syntax UID                 UI: JPEG Baseline (Process 1)
(0002, 0012) Implementation Class UID            UI: 1.2.276.0.7230010.3.0.3.6.0
(0002, 0013) Implementation Version Name         SH: 'OFFIS_DCMTK_360'
-------------------------------------------------
(0008, 0005) Specific Character Set              CS: 'ISO_IR 100'
(0008, 0016) SOP Class UID                       UI: Secondary Capture Image Storage
(0008, 0018) SOP Instance UID                    UI: 1.2.276.0.7230010.3.1.4.8323329.28530.1517874485.775526
(0008, 0020) Study Date                          DA: '19010101'
(0008, 0030) Study Time                          TM: '000000.00'
(0008, 0050) Accession Number                    SH: ''
(0008, 0060) Modality                            CS: 'CR'
(0008, 0064) Conversion Type                     CS: 'WSD'
(0008, 0090) Referring Physician's Name          PN: ''
(0008, 103e) Series Description                  LO: 'view: PA'
(0010, 0010) Patient's Name                      PN: '0004cfab-14fd-4e49-80ba-63a80b6bddd6'
(0010, 0020) Patient ID                          LO: '0004cfab-14fd-4e49-80ba-63a80b6bddd6'
(0010, 0030) Patient's Birth Date                DA: ''
(0010, 0040) Patient's Sex                       CS: 'F'
(0010, 1010) Patient's Age                       AS: '51'
(0018, 0015) Body Part Examined                  CS: 'CHEST'
(0018, 5101) View Position                       CS: 'PA'
(0020, 000d) Study Instance UID                  UI: 1.2.276.0.7230010.3.1.2.8323329.28530.1517874485.775525
(0020, 000e) Series Instance UID                 UI: 1.2.276.0.7230010.3.1.3.8323329.28530.1517874485.775524
(0020, 0010) Study ID                            SH: ''
(0020, 0011) Series Number                       IS: "1"
(0020, 0013) Instance Number                     IS: "1"
(0020, 0020) Patient Orientation                 CS: ''
(0028, 0002) Samples per Pixel                   US: 1
(0028, 0004) Photometric Interpretation          CS: 'MONOCHROME2'
(0028, 0010) Rows                                US: 1024
(0028, 0011) Columns                             US: 1024
(0028, 0030) Pixel Spacing                       DS: [0.14300000000000002, 0.14300000000000002]
(0028, 0100) Bits Allocated                      US: 8
(0028, 0101) Bits Stored                         US: 8
(0028, 0102) High Bit                            US: 7
(0028, 0103) Pixel Representation                US: 0
(0028, 2110) Lossy Image Compression             CS: '01'
(0028, 2114) Lossy Image Compression Method      CS: 'ISO_10918_1'
(7fe0, 0010) Pixel Data                          OB: Array of 142006 elements

Above we check how dicom metadat looks like

In [ ]:
<class 'numpy.ndarray'>
uint8
(1024, 1024)

Also we see that above, our images are numpy arrays with dimension 1024x1024

These dimensions are perfect for training model as they are AI friendly, not extremely high dimensiona nor extremely pixalated

This saves us from image transformation or any sort of data compression

In [ ]:
In [ ]:
In [ ]:

Above we visualise the images along with basic dicom metadat

We see positive class as well as negative class and also with binding boxes

One thing that is evident above is the opacity is not visually identifyable

In [ ]:
In [ ]:
In [ ]:

Next we make a block of code to get all the positive samples into a dictionary.

In [ ]:
n train samples 24154
n valid samples 2560
Total train images: 26714
Images with pneumonia: 6012

Next we split into train and validation

In [ ]:

Next we make a generator class whihc has load data, get data and predict data functions

In [ ]:

Next we write functions to downsample the data(Since positive class is very less than negative class), create resnet layers and complete the network.

In [ ]:

Next we write functions to get jaccord loss metric

In [ ]:
WARNING:tensorflow:From /tensorflow-1.15.2/python3.7/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
Add input shape: (?, 64, 64, 32)
Resnet block input shape: (?, 64, 64, 32)
Resnet block out shape: (?, 64, 64, 32)
concat block out shape: (?, 64, 64, 64)
mixed block out shape: (?, 64, 64, 32)
Add input shape: (?, 64, 64, 32)
Resnet block input shape: (?, 64, 64, 32)
Resnet block out shape: (?, 64, 64, 32)
concat block out shape: (?, 64, 64, 64)
mixed block out shape: (?, 64, 64, 32)
Add input shape: (?, 32, 32, 64)
Resnet block input shape: (?, 32, 32, 64)
Resnet block out shape: (?, 32, 32, 64)
concat block out shape: (?, 32, 32, 128)
mixed block out shape: (?, 32, 32, 64)
Add input shape: (?, 32, 32, 64)
Resnet block input shape: (?, 32, 32, 64)
Resnet block out shape: (?, 32, 32, 64)
concat block out shape: (?, 32, 32, 128)
mixed block out shape: (?, 32, 32, 64)
Add input shape: (?, 16, 16, 128)
Resnet block input shape: (?, 16, 16, 128)
Resnet block out shape: (?, 16, 16, 128)
concat block out shape: (?, 16, 16, 256)
mixed block out shape: (?, 16, 16, 128)
Add input shape: (?, 16, 16, 128)
Resnet block input shape: (?, 16, 16, 128)
Resnet block out shape: (?, 16, 16, 128)
concat block out shape: (?, 16, 16, 256)
mixed block out shape: (?, 16, 16, 128)
Model: "model"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 128, 128, 1) 0                                            
__________________________________________________________________________________________________
conv2d (Conv2D)                 (None, 128, 128, 16) 144         input_1[0][0]                    
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 128, 128, 16) 64          conv2d[0][0]                     
__________________________________________________________________________________________________
leaky_re_lu (LeakyReLU)         (None, 128, 128, 16) 0           batch_normalization[0][0]        
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, 128, 128, 32) 512         leaky_re_lu[0][0]                
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D)    (None, 64, 64, 32)   0           conv2d_1[0][0]                   
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 64, 64, 32)   128         max_pooling2d[0][0]              
__________________________________________________________________________________________________
leaky_re_lu_1 (LeakyReLU)       (None, 64, 64, 32)   0           batch_normalization_1[0][0]      
__________________________________________________________________________________________________
conv2d_2 (Conv2D)               (None, 64, 64, 32)   9216        leaky_re_lu_1[0][0]              
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 64, 64, 32)   128         conv2d_2[0][0]                   
__________________________________________________________________________________________________
leaky_re_lu_2 (LeakyReLU)       (None, 64, 64, 32)   0           batch_normalization_2[0][0]      
__________________________________________________________________________________________________
conv2d_3 (Conv2D)               (None, 64, 64, 32)   9216        leaky_re_lu_2[0][0]              
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 64, 64, 32)   128         conv2d_3[0][0]                   
__________________________________________________________________________________________________
leaky_re_lu_3 (LeakyReLU)       (None, 64, 64, 32)   0           batch_normalization_3[0][0]      
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, 64, 64, 32)   9216        leaky_re_lu_3[0][0]              
__________________________________________________________________________________________________
add (Add)                       (None, 64, 64, 32)   0           conv2d_4[0][0]                   
                                                                 max_pooling2d[0][0]              
__________________________________________________________________________________________________
concatenate (Concatenate)       (None, 64, 64, 64)   0           add[0][0]                        
                                                                 conv2d_4[0][0]                   
__________________________________________________________________________________________________
conv2d_5 (Conv2D)               (None, 64, 64, 32)   2048        concatenate[0][0]                
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 64, 64, 32)   128         conv2d_5[0][0]                   
__________________________________________________________________________________________________
leaky_re_lu_4 (LeakyReLU)       (None, 64, 64, 32)   0           batch_normalization_4[0][0]      
__________________________________________________________________________________________________
conv2d_6 (Conv2D)               (None, 64, 64, 32)   9216        leaky_re_lu_4[0][0]              
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 64, 64, 32)   128         conv2d_6[0][0]                   
__________________________________________________________________________________________________
leaky_re_lu_5 (LeakyReLU)       (None, 64, 64, 32)   0           batch_normalization_5[0][0]      
__________________________________________________________________________________________________
conv2d_7 (Conv2D)               (None, 64, 64, 32)   9216        leaky_re_lu_5[0][0]              
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 64, 64, 32)   128         conv2d_7[0][0]                   
__________________________________________________________________________________________________
leaky_re_lu_6 (LeakyReLU)       (None, 64, 64, 32)   0           batch_normalization_6[0][0]      
__________________________________________________________________________________________________
conv2d_8 (Conv2D)               (None, 64, 64, 32)   9216        leaky_re_lu_6[0][0]              
__________________________________________________________________________________________________
add_1 (Add)                     (None, 64, 64, 32)   0           conv2d_8[0][0]                   
                                                                 conv2d_5[0][0]                   
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 64, 64, 64)   0           add_1[0][0]                      
                                                                 conv2d_8[0][0]                   
__________________________________________________________________________________________________
conv2d_9 (Conv2D)               (None, 64, 64, 32)   2048        concatenate_1[0][0]              
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, 64, 64, 32)   128         conv2d_9[0][0]                   
__________________________________________________________________________________________________
leaky_re_lu_7 (LeakyReLU)       (None, 64, 64, 32)   0           batch_normalization_7[0][0]      
__________________________________________________________________________________________________
conv2d_10 (Conv2D)              (None, 64, 64, 64)   2048        leaky_re_lu_7[0][0]              
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D)  (None, 32, 32, 64)   0           conv2d_10[0][0]                  
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, 32, 32, 64)   256         max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
leaky_re_lu_8 (LeakyReLU)       (None, 32, 32, 64)   0           batch_normalization_8[0][0]      
__________________________________________________________________________________________________
conv2d_11 (Conv2D)              (None, 32, 32, 64)   36864       leaky_re_lu_8[0][0]              
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, 32, 32, 64)   256         conv2d_11[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_9 (LeakyReLU)       (None, 32, 32, 64)   0           batch_normalization_9[0][0]      
__________________________________________________________________________________________________
conv2d_12 (Conv2D)              (None, 32, 32, 64)   36864       leaky_re_lu_9[0][0]              
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, 32, 32, 64)   256         conv2d_12[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_10 (LeakyReLU)      (None, 32, 32, 64)   0           batch_normalization_10[0][0]     
__________________________________________________________________________________________________
conv2d_13 (Conv2D)              (None, 32, 32, 64)   36864       leaky_re_lu_10[0][0]             
__________________________________________________________________________________________________
add_2 (Add)                     (None, 32, 32, 64)   0           conv2d_13[0][0]                  
                                                                 max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
concatenate_2 (Concatenate)     (None, 32, 32, 128)  0           add_2[0][0]                      
                                                                 conv2d_13[0][0]                  
__________________________________________________________________________________________________
conv2d_14 (Conv2D)              (None, 32, 32, 64)   8192        concatenate_2[0][0]              
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, 32, 32, 64)   256         conv2d_14[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_11 (LeakyReLU)      (None, 32, 32, 64)   0           batch_normalization_11[0][0]     
__________________________________________________________________________________________________
conv2d_15 (Conv2D)              (None, 32, 32, 64)   36864       leaky_re_lu_11[0][0]             
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, 32, 32, 64)   256         conv2d_15[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_12 (LeakyReLU)      (None, 32, 32, 64)   0           batch_normalization_12[0][0]     
__________________________________________________________________________________________________
conv2d_16 (Conv2D)              (None, 32, 32, 64)   36864       leaky_re_lu_12[0][0]             
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, 32, 32, 64)   256         conv2d_16[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_13 (LeakyReLU)      (None, 32, 32, 64)   0           batch_normalization_13[0][0]     
__________________________________________________________________________________________________
conv2d_17 (Conv2D)              (None, 32, 32, 64)   36864       leaky_re_lu_13[0][0]             
__________________________________________________________________________________________________
add_3 (Add)                     (None, 32, 32, 64)   0           conv2d_17[0][0]                  
                                                                 conv2d_14[0][0]                  
__________________________________________________________________________________________________
concatenate_3 (Concatenate)     (None, 32, 32, 128)  0           add_3[0][0]                      
                                                                 conv2d_17[0][0]                  
__________________________________________________________________________________________________
conv2d_18 (Conv2D)              (None, 32, 32, 64)   8192        concatenate_3[0][0]              
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, 32, 32, 64)   256         conv2d_18[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_14 (LeakyReLU)      (None, 32, 32, 64)   0           batch_normalization_14[0][0]     
__________________________________________________________________________________________________
conv2d_19 (Conv2D)              (None, 32, 32, 128)  8192        leaky_re_lu_14[0][0]             
__________________________________________________________________________________________________
max_pooling2d_2 (MaxPooling2D)  (None, 16, 16, 128)  0           conv2d_19[0][0]                  
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, 16, 16, 128)  512         max_pooling2d_2[0][0]            
__________________________________________________________________________________________________
leaky_re_lu_15 (LeakyReLU)      (None, 16, 16, 128)  0           batch_normalization_15[0][0]     
__________________________________________________________________________________________________
conv2d_20 (Conv2D)              (None, 16, 16, 128)  147456      leaky_re_lu_15[0][0]             
__________________________________________________________________________________________________
batch_normalization_16 (BatchNo (None, 16, 16, 128)  512         conv2d_20[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_16 (LeakyReLU)      (None, 16, 16, 128)  0           batch_normalization_16[0][0]     
__________________________________________________________________________________________________
conv2d_21 (Conv2D)              (None, 16, 16, 128)  147456      leaky_re_lu_16[0][0]             
__________________________________________________________________________________________________
batch_normalization_17 (BatchNo (None, 16, 16, 128)  512         conv2d_21[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_17 (LeakyReLU)      (None, 16, 16, 128)  0           batch_normalization_17[0][0]     
__________________________________________________________________________________________________
conv2d_22 (Conv2D)              (None, 16, 16, 128)  147456      leaky_re_lu_17[0][0]             
__________________________________________________________________________________________________
add_4 (Add)                     (None, 16, 16, 128)  0           conv2d_22[0][0]                  
                                                                 max_pooling2d_2[0][0]            
__________________________________________________________________________________________________
concatenate_4 (Concatenate)     (None, 16, 16, 256)  0           add_4[0][0]                      
                                                                 conv2d_22[0][0]                  
__________________________________________________________________________________________________
conv2d_23 (Conv2D)              (None, 16, 16, 128)  32768       concatenate_4[0][0]              
__________________________________________________________________________________________________
batch_normalization_18 (BatchNo (None, 16, 16, 128)  512         conv2d_23[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_18 (LeakyReLU)      (None, 16, 16, 128)  0           batch_normalization_18[0][0]     
__________________________________________________________________________________________________
conv2d_24 (Conv2D)              (None, 16, 16, 128)  147456      leaky_re_lu_18[0][0]             
__________________________________________________________________________________________________
batch_normalization_19 (BatchNo (None, 16, 16, 128)  512         conv2d_24[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_19 (LeakyReLU)      (None, 16, 16, 128)  0           batch_normalization_19[0][0]     
__________________________________________________________________________________________________
conv2d_25 (Conv2D)              (None, 16, 16, 128)  147456      leaky_re_lu_19[0][0]             
__________________________________________________________________________________________________
batch_normalization_20 (BatchNo (None, 16, 16, 128)  512         conv2d_25[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_20 (LeakyReLU)      (None, 16, 16, 128)  0           batch_normalization_20[0][0]     
__________________________________________________________________________________________________
conv2d_26 (Conv2D)              (None, 16, 16, 128)  147456      leaky_re_lu_20[0][0]             
__________________________________________________________________________________________________
add_5 (Add)                     (None, 16, 16, 128)  0           conv2d_26[0][0]                  
                                                                 conv2d_23[0][0]                  
__________________________________________________________________________________________________
concatenate_5 (Concatenate)     (None, 16, 16, 256)  0           add_5[0][0]                      
                                                                 conv2d_26[0][0]                  
__________________________________________________________________________________________________
conv2d_27 (Conv2D)              (None, 16, 16, 128)  32768       concatenate_5[0][0]              
__________________________________________________________________________________________________
batch_normalization_21 (BatchNo (None, 16, 16, 128)  512         conv2d_27[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_21 (LeakyReLU)      (None, 16, 16, 128)  0           batch_normalization_21[0][0]     
__________________________________________________________________________________________________
conv2d_28 (Conv2D)              (None, 16, 16, 1)    129         leaky_re_lu_21[0][0]             
__________________________________________________________________________________________________
up_sampling2d (UpSampling2D)    (None, 128, 128, 1)  0           conv2d_28[0][0]                  
==================================================================================================
Total params: 1,264,593
Trainable params: 1,261,425
Non-trainable params: 3,168
__________________________________________________________________________________________________
model summary: None

We make a model with 28 Convolution layers with clubbed with leaky relu and batch normalisation layers

In [ ]:
Epoch 1/10
WARNING:tensorflow:From /tensorflow-1.15.2/python3.7/tensorflow_core/python/ops/math_grad.py:1424: where
(from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
1507/1507 - 2646s - loss: 0.4735 - accuracy: 0.8636 - mean_iou: 0.6307 - val_loss: 0.6701 - val_accuracy: 0.6345 - val_mean_iou: 0.2398 - lr: 0.0010
Epoch 2/10
1507/1507 - 2277s - loss: 0.4450 - accuracy: 0.8668 - mean_iou: 0.6784 - val_loss: 0.4906 - val_accuracy: 0.6479 - val_mean_iou: 0.7674 - lr: 9.9606e-04
Epoch 3/10
1507/1507 - 2515s - loss: 0.4344 - accuracy: 0.9363 - mean_iou: 0.6933 - val_loss: 0.4465 - val_accuracy: 0.7194 - val_mean_iou: 0.7169 - lr: 9.8429e-04
Epoch 4/10
1507/1507 - 2618s - loss: 0.4286 - accuracy: 0.9385 - mean_iou: 0.6959 - val_loss: 0.4256 - val_accuracy: 0.7298 - val_mean_iou: 0.7254 - lr: 9.6489e-04
Epoch 5/10
1507/1507 - 2469s - loss: 0.4217 - accuracy: 0.9497 - mean_iou: 0.7069 - val_loss: 0.4358 - val_accuracy: 0.7421 - val_mean_iou: 0.7355 - lr: 9.3815e-04
Epoch 6/10
1507/1507 - 2573s - loss: 0.4173 - accuracy: 0.9400 - mean_iou: 0.7084 - val_loss: 0.4175 - val_accuracy: 0.7599 - val_mean_iou: 0.7172 - lr: 9.0451e-04
Epoch 7/10
1507/1507 - 2478s - loss: 0.4121 - accuracy: 0.9506 - mean_iou: 0.7106 - val_loss: 0.4145 - val_accuracy: 0.7731 - val_mean_iou: 0.7359 - lr: 8.6448e-04
Epoch 8/10
1507/1507 - 2290s - loss: 0.4099 - accuracy: 0.9609 - mean_iou: 0.7114 - val_loss: 0.4129 - val_accuracy: 0.7821 - val_mean_iou: 0.7054 - lr: 8.1871e-04
Epoch 9/10
1507/1507 - 2263s - loss: 0.4067 - accuracy: 0.9716 - mean_iou: 0.7151 - val_loss: 0.4147 - val_accuracy: 0.8019 - val_mean_iou: 0.7057 - lr: 7.6791e-04
Epoch 10/10
1507/1507 - 2262s - loss: 0.4044 - accuracy: 0.9716 - mean_iou: 0.7196 - val_loss: 0.4095 - val_accuracy: 0.8391 - val_mean_iou: 0.7293 - lr: 7.1289e-04
In [ ]:
Test accuracy: 0.8124

Next we train and test the model and see we have got Val accuracy of 0.8391 and test accuracy of 0.8124 whihch is not bad for scracth trained model

Next we try our hands on transfer learning by using pre-trained models.

First we try VGG 19

In [ ]:
In [ ]:
In [ ]:
In [ ]:
In [ ]:
In [4]:
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/vgg19/vgg19_weights_tf_dim_ordering_tf_kernels_notop.h5
80142336/80134624 [==============================] - 1s 0us/step
last layer output shape:  (None, 8, 8, 512)
In [5]:
In [6]:
Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         [(None, 256, 256, 3)]     0         
_________________________________________________________________
block1_conv1 (Conv2D)        (None, 256, 256, 64)      1792      
_________________________________________________________________
block1_conv2 (Conv2D)        (None, 256, 256, 64)      36928     
_________________________________________________________________
block1_pool (MaxPooling2D)   (None, 128, 128, 64)      0         
_________________________________________________________________
block2_conv1 (Conv2D)        (None, 128, 128, 128)     73856     
_________________________________________________________________
block2_conv2 (Conv2D)        (None, 128, 128, 128)     147584    
_________________________________________________________________
block2_pool (MaxPooling2D)   (None, 64, 64, 128)       0         
_________________________________________________________________
block3_conv1 (Conv2D)        (None, 64, 64, 256)       295168    
_________________________________________________________________
block3_conv2 (Conv2D)        (None, 64, 64, 256)       590080    
_________________________________________________________________
block3_conv3 (Conv2D)        (None, 64, 64, 256)       590080    
_________________________________________________________________
block3_conv4 (Conv2D)        (None, 64, 64, 256)       590080    
_________________________________________________________________
block3_pool (MaxPooling2D)   (None, 32, 32, 256)       0         
_________________________________________________________________
block4_conv1 (Conv2D)        (None, 32, 32, 512)       1180160   
_________________________________________________________________
block4_conv2 (Conv2D)        (None, 32, 32, 512)       2359808   
_________________________________________________________________
block4_conv3 (Conv2D)        (None, 32, 32, 512)       2359808   
_________________________________________________________________
block4_conv4 (Conv2D)        (None, 32, 32, 512)       2359808   
_________________________________________________________________
block4_pool (MaxPooling2D)   (None, 16, 16, 512)       0         
_________________________________________________________________
block5_conv1 (Conv2D)        (None, 16, 16, 512)       2359808   
_________________________________________________________________
block5_conv2 (Conv2D)        (None, 16, 16, 512)       2359808   
_________________________________________________________________
block5_conv3 (Conv2D)        (None, 16, 16, 512)       2359808   
_________________________________________________________________
block5_conv4 (Conv2D)        (None, 16, 16, 512)       2359808   
_________________________________________________________________
block5_pool (MaxPooling2D)   (None, 8, 8, 512)         0         
_________________________________________________________________
flatten (Flatten)            (None, 32768)             0         
_________________________________________________________________
dense (Dense)                (None, 1024)              33555456  
_________________________________________________________________
leaky_re_lu (LeakyReLU)      (None, 1024)              0         
_________________________________________________________________
dropout (Dropout)            (None, 1024)              0         
_________________________________________________________________
batch_normalization (BatchNo (None, 1024)              4096      
_________________________________________________________________
dense_1 (Dense)              (None, 1024)              1049600   
_________________________________________________________________
leaky_re_lu_1 (LeakyReLU)    (None, 1024)              0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 1024)              0         
_________________________________________________________________
batch_normalization_1 (Batch (None, 1024)              4096      
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 1025      
=================================================================
Total params: 54,638,657
Trainable params: 34,610,177
Non-trainable params: 20,028,480
_________________________________________________________________
In [ ]:
In [ ]:
Epoch 1/10
100/100 [==============================] - 7985s 162s/step - loss: 1.2630 - accuracy: 0.5153 - val_loss: 0.8171 - val_accuracy: 0.2253
Epoch 2/10
100/100 [==============================] - 7930s 160s/step - loss: 1.4530 - accuracy: 0.5392 - val_loss: 0.9142 - val_accuracy: 0.2366
Epoch 3/10
100/100 [==============================] - 7928s 163s/step - loss: 1.3823 - accuracy: 0.5643 - val_loss: 0.6749 - val_accuracy: 0.3747
Epoch 4/10
100/100 [==============================] - 7900s 160s/step - loss: 1.4660 - accuracy: 0.6153 - val_loss: 0.6178 - val_accuracy: 0.4153
Epoch 5/10
100/100 [==============================] - 7840s 159s/step - loss: 1.4550 - accuracy: 0.6891 - val_loss: 0.6744 - val_accuracy: 0.5561
Epoch 6/10
100/100 [==============================] - 7936s 161s/step - loss: 1.4023 - accuracy: 0.6949 - val_loss: 0.6045 - val_accuracy: 0.6087
Epoch 7/10
100/100 [==============================] - 7988s 160s/step - loss: 1.3342 - accuracy: 0.7183 - val_loss: 0.7171 - val_accuracy: 0.6553
Epoch 8/10
100/100 [==============================] - 7990s 161s/step - loss: 1.2510 - accuracy: 0.8812 - val_loss: 0.8172 - val_accuracy: 0.6643
Epoch 9/10
100/100 [==============================] - 7778s 161s/step - loss: 1.2823 - accuracy: 0.9043 - val_loss: 0.6849 - val_accuracy: 0.7047
Epoch 10/10
100/100 [==============================] - 7828s 156s/step - loss: 1.2243 - accuracy: 0.9125 - val_loss: 0.6089 - val_accuracy: 0.7447
Epoch 00003: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.
In [ ]:
Test accuracy: 0.7016

VGG19 gives us a test accuracy of 70%

Next we try Inception v3

In [7]:
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/inception_v3/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5
87916544/87910968 [==============================] - 1s 0us/step
Model: "inception_v3"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_2 (InputLayer)            [(None, 256, 256, 3) 0                                            
__________________________________________________________________________________________________
conv2d (Conv2D)                 (None, 127, 127, 32) 864         input_2[0][0]                    
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 127, 127, 32) 96          conv2d[0][0]                     
__________________________________________________________________________________________________
activation (Activation)         (None, 127, 127, 32) 0           batch_normalization_2[0][0]      
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, 125, 125, 32) 9216        activation[0][0]                 
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 125, 125, 32) 96          conv2d_1[0][0]                   
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 125, 125, 32) 0           batch_normalization_3[0][0]      
__________________________________________________________________________________________________
conv2d_2 (Conv2D)               (None, 125, 125, 64) 18432       activation_1[0][0]               
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 125, 125, 64) 192         conv2d_2[0][0]                   
__________________________________________________________________________________________________
activation_2 (Activation)       (None, 125, 125, 64) 0           batch_normalization_4[0][0]      
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D)    (None, 62, 62, 64)   0           activation_2[0][0]               
__________________________________________________________________________________________________
conv2d_3 (Conv2D)               (None, 62, 62, 80)   5120        max_pooling2d[0][0]              
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 62, 62, 80)   240         conv2d_3[0][0]                   
__________________________________________________________________________________________________
activation_3 (Activation)       (None, 62, 62, 80)   0           batch_normalization_5[0][0]      
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, 60, 60, 192)  138240      activation_3[0][0]               
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 60, 60, 192)  576         conv2d_4[0][0]                   
__________________________________________________________________________________________________
activation_4 (Activation)       (None, 60, 60, 192)  0           batch_normalization_6[0][0]      
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D)  (None, 29, 29, 192)  0           activation_4[0][0]               
__________________________________________________________________________________________________
conv2d_8 (Conv2D)               (None, 29, 29, 64)   12288       max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, 29, 29, 64)   192         conv2d_8[0][0]                   
__________________________________________________________________________________________________
activation_8 (Activation)       (None, 29, 29, 64)   0           batch_normalization_10[0][0]     
__________________________________________________________________________________________________
conv2d_6 (Conv2D)               (None, 29, 29, 48)   9216        max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
conv2d_9 (Conv2D)               (None, 29, 29, 96)   55296       activation_8[0][0]               
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, 29, 29, 48)   144         conv2d_6[0][0]                   
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, 29, 29, 96)   288         conv2d_9[0][0]                   
__________________________________________________________________________________________________
activation_6 (Activation)       (None, 29, 29, 48)   0           batch_normalization_8[0][0]      
__________________________________________________________________________________________________
activation_9 (Activation)       (None, 29, 29, 96)   0           batch_normalization_11[0][0]     
__________________________________________________________________________________________________
average_pooling2d (AveragePooli (None, 29, 29, 192)  0           max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
conv2d_5 (Conv2D)               (None, 29, 29, 64)   12288       max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
conv2d_7 (Conv2D)               (None, 29, 29, 64)   76800       activation_6[0][0]               
__________________________________________________________________________________________________
conv2d_10 (Conv2D)              (None, 29, 29, 96)   82944       activation_9[0][0]               
__________________________________________________________________________________________________
conv2d_11 (Conv2D)              (None, 29, 29, 32)   6144        average_pooling2d[0][0]          
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, 29, 29, 64)   192         conv2d_5[0][0]                   
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, 29, 29, 64)   192         conv2d_7[0][0]                   
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, 29, 29, 96)   288         conv2d_10[0][0]                  
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, 29, 29, 32)   96          conv2d_11[0][0]                  
__________________________________________________________________________________________________
activation_5 (Activation)       (None, 29, 29, 64)   0           batch_normalization_7[0][0]      
__________________________________________________________________________________________________
activation_7 (Activation)       (None, 29, 29, 64)   0           batch_normalization_9[0][0]      
__________________________________________________________________________________________________
activation_10 (Activation)      (None, 29, 29, 96)   0           batch_normalization_12[0][0]     
__________________________________________________________________________________________________
activation_11 (Activation)      (None, 29, 29, 32)   0           batch_normalization_13[0][0]     
__________________________________________________________________________________________________
mixed0 (Concatenate)            (None, 29, 29, 256)  0           activation_5[0][0]               
                                                                 activation_7[0][0]               
                                                                 activation_10[0][0]              
                                                                 activation_11[0][0]              
__________________________________________________________________________________________________
conv2d_15 (Conv2D)              (None, 29, 29, 64)   16384       mixed0[0][0]                     
__________________________________________________________________________________________________
batch_normalization_17 (BatchNo (None, 29, 29, 64)   192         conv2d_15[0][0]                  
__________________________________________________________________________________________________
activation_15 (Activation)      (None, 29, 29, 64)   0           batch_normalization_17[0][0]     
__________________________________________________________________________________________________
conv2d_13 (Conv2D)              (None, 29, 29, 48)   12288       mixed0[0][0]                     
__________________________________________________________________________________________________
conv2d_16 (Conv2D)              (None, 29, 29, 96)   55296       activation_15[0][0]              
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, 29, 29, 48)   144         conv2d_13[0][0]                  
__________________________________________________________________________________________________
batch_normalization_18 (BatchNo (None, 29, 29, 96)   288         conv2d_16[0][0]                  
__________________________________________________________________________________________________
activation_13 (Activation)      (None, 29, 29, 48)   0           batch_normalization_15[0][0]     
__________________________________________________________________________________________________
activation_16 (Activation)      (None, 29, 29, 96)   0           batch_normalization_18[0][0]     
__________________________________________________________________________________________________
average_pooling2d_1 (AveragePoo (None, 29, 29, 256)  0           mixed0[0][0]                     
__________________________________________________________________________________________________
conv2d_12 (Conv2D)              (None, 29, 29, 64)   16384       mixed0[0][0]                     
__________________________________________________________________________________________________
conv2d_14 (Conv2D)              (None, 29, 29, 64)   76800       activation_13[0][0]              
__________________________________________________________________________________________________
conv2d_17 (Conv2D)              (None, 29, 29, 96)   82944       activation_16[0][0]              
__________________________________________________________________________________________________
conv2d_18 (Conv2D)              (None, 29, 29, 64)   16384       average_pooling2d_1[0][0]        
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, 29, 29, 64)   192         conv2d_12[0][0]                  
__________________________________________________________________________________________________
batch_normalization_16 (BatchNo (None, 29, 29, 64)   192         conv2d_14[0][0]                  
__________________________________________________________________________________________________
batch_normalization_19 (BatchNo (None, 29, 29, 96)   288         conv2d_17[0][0]                  
__________________________________________________________________________________________________
batch_normalization_20 (BatchNo (None, 29, 29, 64)   192         conv2d_18[0][0]                  
__________________________________________________________________________________________________
activation_12 (Activation)      (None, 29, 29, 64)   0           batch_normalization_14[0][0]     
__________________________________________________________________________________________________
activation_14 (Activation)      (None, 29, 29, 64)   0           batch_normalization_16[0][0]     
__________________________________________________________________________________________________
activation_17 (Activation)      (None, 29, 29, 96)   0           batch_normalization_19[0][0]     
__________________________________________________________________________________________________
activation_18 (Activation)      (None, 29, 29, 64)   0           batch_normalization_20[0][0]     
__________________________________________________________________________________________________
mixed1 (Concatenate)            (None, 29, 29, 288)  0           activation_12[0][0]              
                                                                 activation_14[0][0]              
                                                                 activation_17[0][0]              
                                                                 activation_18[0][0]              
__________________________________________________________________________________________________
conv2d_22 (Conv2D)              (None, 29, 29, 64)   18432       mixed1[0][0]                     
__________________________________________________________________________________________________
batch_normalization_24 (BatchNo (None, 29, 29, 64)   192         conv2d_22[0][0]                  
__________________________________________________________________________________________________
activation_22 (Activation)      (None, 29, 29, 64)   0           batch_normalization_24[0][0]     
__________________________________________________________________________________________________
conv2d_20 (Conv2D)              (None, 29, 29, 48)   13824       mixed1[0][0]                     
__________________________________________________________________________________________________
conv2d_23 (Conv2D)              (None, 29, 29, 96)   55296       activation_22[0][0]              
__________________________________________________________________________________________________
batch_normalization_22 (BatchNo (None, 29, 29, 48)   144         conv2d_20[0][0]                  
__________________________________________________________________________________________________
batch_normalization_25 (BatchNo (None, 29, 29, 96)   288         conv2d_23[0][0]                  
__________________________________________________________________________________________________
activation_20 (Activation)      (None, 29, 29, 48)   0           batch_normalization_22[0][0]     
__________________________________________________________________________________________________
activation_23 (Activation)      (None, 29, 29, 96)   0           batch_normalization_25[0][0]     
__________________________________________________________________________________________________
average_pooling2d_2 (AveragePoo (None, 29, 29, 288)  0           mixed1[0][0]                     
__________________________________________________________________________________________________
conv2d_19 (Conv2D)              (None, 29, 29, 64)   18432       mixed1[0][0]                     
__________________________________________________________________________________________________
conv2d_21 (Conv2D)              (None, 29, 29, 64)   76800       activation_20[0][0]              
__________________________________________________________________________________________________
conv2d_24 (Conv2D)              (None, 29, 29, 96)   82944       activation_23[0][0]              
__________________________________________________________________________________________________
conv2d_25 (Conv2D)              (None, 29, 29, 64)   18432       average_pooling2d_2[0][0]        
__________________________________________________________________________________________________
batch_normalization_21 (BatchNo (None, 29, 29, 64)   192         conv2d_19[0][0]                  
__________________________________________________________________________________________________
batch_normalization_23 (BatchNo (None, 29, 29, 64)   192         conv2d_21[0][0]                  
__________________________________________________________________________________________________
batch_normalization_26 (BatchNo (None, 29, 29, 96)   288         conv2d_24[0][0]                  
__________________________________________________________________________________________________
batch_normalization_27 (BatchNo (None, 29, 29, 64)   192         conv2d_25[0][0]                  
__________________________________________________________________________________________________
activation_19 (Activation)      (None, 29, 29, 64)   0           batch_normalization_21[0][0]     
__________________________________________________________________________________________________
activation_21 (Activation)      (None, 29, 29, 64)   0           batch_normalization_23[0][0]     
__________________________________________________________________________________________________
activation_24 (Activation)      (None, 29, 29, 96)   0           batch_normalization_26[0][0]     
__________________________________________________________________________________________________
activation_25 (Activation)      (None, 29, 29, 64)   0           batch_normalization_27[0][0]     
__________________________________________________________________________________________________
mixed2 (Concatenate)            (None, 29, 29, 288)  0           activation_19[0][0]              
                                                                 activation_21[0][0]              
                                                                 activation_24[0][0]              
                                                                 activation_25[0][0]              
__________________________________________________________________________________________________
conv2d_27 (Conv2D)              (None, 29, 29, 64)   18432       mixed2[0][0]                     
__________________________________________________________________________________________________
batch_normalization_29 (BatchNo (None, 29, 29, 64)   192         conv2d_27[0][0]                  
__________________________________________________________________________________________________
activation_27 (Activation)      (None, 29, 29, 64)   0           batch_normalization_29[0][0]     
__________________________________________________________________________________________________
conv2d_28 (Conv2D)              (None, 29, 29, 96)   55296       activation_27[0][0]              
__________________________________________________________________________________________________
batch_normalization_30 (BatchNo (None, 29, 29, 96)   288         conv2d_28[0][0]                  
__________________________________________________________________________________________________
activation_28 (Activation)      (None, 29, 29, 96)   0           batch_normalization_30[0][0]     
__________________________________________________________________________________________________
conv2d_26 (Conv2D)              (None, 14, 14, 384)  995328      mixed2[0][0]                     
__________________________________________________________________________________________________
conv2d_29 (Conv2D)              (None, 14, 14, 96)   82944       activation_28[0][0]              
__________________________________________________________________________________________________
batch_normalization_28 (BatchNo (None, 14, 14, 384)  1152        conv2d_26[0][0]                  
__________________________________________________________________________________________________
batch_normalization_31 (BatchNo (None, 14, 14, 96)   288         conv2d_29[0][0]                  
__________________________________________________________________________________________________
activation_26 (Activation)      (None, 14, 14, 384)  0           batch_normalization_28[0][0]     
__________________________________________________________________________________________________
activation_29 (Activation)      (None, 14, 14, 96)   0           batch_normalization_31[0][0]     
__________________________________________________________________________________________________
max_pooling2d_2 (MaxPooling2D)  (None, 14, 14, 288)  0           mixed2[0][0]                     
__________________________________________________________________________________________________
mixed3 (Concatenate)            (None, 14, 14, 768)  0           activation_26[0][0]              
                                                                 activation_29[0][0]              
                                                                 max_pooling2d_2[0][0]            
__________________________________________________________________________________________________
conv2d_34 (Conv2D)              (None, 14, 14, 128)  98304       mixed3[0][0]                     
__________________________________________________________________________________________________
batch_normalization_36 (BatchNo (None, 14, 14, 128)  384         conv2d_34[0][0]                  
__________________________________________________________________________________________________
activation_34 (Activation)      (None, 14, 14, 128)  0           batch_normalization_36[0][0]     
__________________________________________________________________________________________________
conv2d_35 (Conv2D)              (None, 14, 14, 128)  114688      activation_34[0][0]              
__________________________________________________________________________________________________
batch_normalization_37 (BatchNo (None, 14, 14, 128)  384         conv2d_35[0][0]                  
__________________________________________________________________________________________________
activation_35 (Activation)      (None, 14, 14, 128)  0           batch_normalization_37[0][0]     
__________________________________________________________________________________________________
conv2d_31 (Conv2D)              (None, 14, 14, 128)  98304       mixed3[0][0]                     
__________________________________________________________________________________________________
conv2d_36 (Conv2D)              (None, 14, 14, 128)  114688      activation_35[0][0]              
__________________________________________________________________________________________________
batch_normalization_33 (BatchNo (None, 14, 14, 128)  384         conv2d_31[0][0]                  
__________________________________________________________________________________________________
batch_normalization_38 (BatchNo (None, 14, 14, 128)  384         conv2d_36[0][0]                  
__________________________________________________________________________________________________
activation_31 (Activation)      (None, 14, 14, 128)  0           batch_normalization_33[0][0]     
__________________________________________________________________________________________________
activation_36 (Activation)      (None, 14, 14, 128)  0           batch_normalization_38[0][0]     
__________________________________________________________________________________________________
conv2d_32 (Conv2D)              (None, 14, 14, 128)  114688      activation_31[0][0]              
__________________________________________________________________________________________________
conv2d_37 (Conv2D)              (None, 14, 14, 128)  114688      activation_36[0][0]              
__________________________________________________________________________________________________
batch_normalization_34 (BatchNo (None, 14, 14, 128)  384         conv2d_32[0][0]                  
__________________________________________________________________________________________________
batch_normalization_39 (BatchNo (None, 14, 14, 128)  384         conv2d_37[0][0]                  
__________________________________________________________________________________________________
activation_32 (Activation)      (None, 14, 14, 128)  0           batch_normalization_34[0][0]     
__________________________________________________________________________________________________
activation_37 (Activation)      (None, 14, 14, 128)  0           batch_normalization_39[0][0]     
__________________________________________________________________________________________________
average_pooling2d_3 (AveragePoo (None, 14, 14, 768)  0           mixed3[0][0]                     
__________________________________________________________________________________________________
conv2d_30 (Conv2D)              (None, 14, 14, 192)  147456      mixed3[0][0]                     
__________________________________________________________________________________________________
conv2d_33 (Conv2D)              (None, 14, 14, 192)  172032      activation_32[0][0]              
__________________________________________________________________________________________________
conv2d_38 (Conv2D)              (None, 14, 14, 192)  172032      activation_37[0][0]              
__________________________________________________________________________________________________
conv2d_39 (Conv2D)              (None, 14, 14, 192)  147456      average_pooling2d_3[0][0]        
__________________________________________________________________________________________________
batch_normalization_32 (BatchNo (None, 14, 14, 192)  576         conv2d_30[0][0]                  
__________________________________________________________________________________________________
batch_normalization_35 (BatchNo (None, 14, 14, 192)  576         conv2d_33[0][0]                  
__________________________________________________________________________________________________
batch_normalization_40 (BatchNo (None, 14, 14, 192)  576         conv2d_38[0][0]                  
__________________________________________________________________________________________________
batch_normalization_41 (BatchNo (None, 14, 14, 192)  576         conv2d_39[0][0]                  
__________________________________________________________________________________________________
activation_30 (Activation)      (None, 14, 14, 192)  0           batch_normalization_32[0][0]     
__________________________________________________________________________________________________
activation_33 (Activation)      (None, 14, 14, 192)  0           batch_normalization_35[0][0]     
__________________________________________________________________________________________________
activation_38 (Activation)      (None, 14, 14, 192)  0           batch_normalization_40[0][0]     
__________________________________________________________________________________________________
activation_39 (Activation)      (None, 14, 14, 192)  0           batch_normalization_41[0][0]     
__________________________________________________________________________________________________
mixed4 (Concatenate)            (None, 14, 14, 768)  0           activation_30[0][0]              
                                                                 activation_33[0][0]              
                                                                 activation_38[0][0]              
                                                                 activation_39[0][0]              
__________________________________________________________________________________________________
conv2d_44 (Conv2D)              (None, 14, 14, 160)  122880      mixed4[0][0]                     
__________________________________________________________________________________________________
batch_normalization_46 (BatchNo (None, 14, 14, 160)  480         conv2d_44[0][0]                  
__________________________________________________________________________________________________
activation_44 (Activation)      (None, 14, 14, 160)  0           batch_normalization_46[0][0]     
__________________________________________________________________________________________________
conv2d_45 (Conv2D)              (None, 14, 14, 160)  179200      activation_44[0][0]              
__________________________________________________________________________________________________
batch_normalization_47 (BatchNo (None, 14, 14, 160)  480         conv2d_45[0][0]                  
__________________________________________________________________________________________________
activation_45 (Activation)      (None, 14, 14, 160)  0           batch_normalization_47[0][0]     
__________________________________________________________________________________________________
conv2d_41 (Conv2D)              (None, 14, 14, 160)  122880      mixed4[0][0]                     
__________________________________________________________________________________________________
conv2d_46 (Conv2D)              (None, 14, 14, 160)  179200      activation_45[0][0]              
__________________________________________________________________________________________________
batch_normalization_43 (BatchNo (None, 14, 14, 160)  480         conv2d_41[0][0]                  
__________________________________________________________________________________________________
batch_normalization_48 (BatchNo (None, 14, 14, 160)  480         conv2d_46[0][0]                  
__________________________________________________________________________________________________
activation_41 (Activation)      (None, 14, 14, 160)  0           batch_normalization_43[0][0]     
__________________________________________________________________________________________________
activation_46 (Activation)      (None, 14, 14, 160)  0           batch_normalization_48[0][0]     
__________________________________________________________________________________________________
conv2d_42 (Conv2D)              (None, 14, 14, 160)  179200      activation_41[0][0]              
__________________________________________________________________________________________________
conv2d_47 (Conv2D)              (None, 14, 14, 160)  179200      activation_46[0][0]              
__________________________________________________________________________________________________
batch_normalization_44 (BatchNo (None, 14, 14, 160)  480         conv2d_42[0][0]                  
__________________________________________________________________________________________________
batch_normalization_49 (BatchNo (None, 14, 14, 160)  480         conv2d_47[0][0]                  
__________________________________________________________________________________________________
activation_42 (Activation)      (None, 14, 14, 160)  0           batch_normalization_44[0][0]     
__________________________________________________________________________________________________
activation_47 (Activation)      (None, 14, 14, 160)  0           batch_normalization_49[0][0]     
__________________________________________________________________________________________________
average_pooling2d_4 (AveragePoo (None, 14, 14, 768)  0           mixed4[0][0]                     
__________________________________________________________________________________________________
conv2d_40 (Conv2D)              (None, 14, 14, 192)  147456      mixed4[0][0]                     
__________________________________________________________________________________________________
conv2d_43 (Conv2D)              (None, 14, 14, 192)  215040      activation_42[0][0]              
__________________________________________________________________________________________________
conv2d_48 (Conv2D)              (None, 14, 14, 192)  215040      activation_47[0][0]              
__________________________________________________________________________________________________
conv2d_49 (Conv2D)              (None, 14, 14, 192)  147456      average_pooling2d_4[0][0]        
__________________________________________________________________________________________________
batch_normalization_42 (BatchNo (None, 14, 14, 192)  576         conv2d_40[0][0]                  
__________________________________________________________________________________________________
batch_normalization_45 (BatchNo (None, 14, 14, 192)  576         conv2d_43[0][0]                  
__________________________________________________________________________________________________
batch_normalization_50 (BatchNo (None, 14, 14, 192)  576         conv2d_48[0][0]                  
__________________________________________________________________________________________________
batch_normalization_51 (BatchNo (None, 14, 14, 192)  576         conv2d_49[0][0]                  
__________________________________________________________________________________________________
activation_40 (Activation)      (None, 14, 14, 192)  0           batch_normalization_42[0][0]     
__________________________________________________________________________________________________
activation_43 (Activation)      (None, 14, 14, 192)  0           batch_normalization_45[0][0]     
__________________________________________________________________________________________________
activation_48 (Activation)      (None, 14, 14, 192)  0           batch_normalization_50[0][0]     
__________________________________________________________________________________________________
activation_49 (Activation)      (None, 14, 14, 192)  0           batch_normalization_51[0][0]     
__________________________________________________________________________________________________
mixed5 (Concatenate)            (None, 14, 14, 768)  0           activation_40[0][0]              
                                                                 activation_43[0][0]              
                                                                 activation_48[0][0]              
                                                                 activation_49[0][0]              
__________________________________________________________________________________________________
conv2d_54 (Conv2D)              (None, 14, 14, 160)  122880      mixed5[0][0]                     
__________________________________________________________________________________________________
batch_normalization_56 (BatchNo (None, 14, 14, 160)  480         conv2d_54[0][0]                  
__________________________________________________________________________________________________
activation_54 (Activation)      (None, 14, 14, 160)  0           batch_normalization_56[0][0]     
__________________________________________________________________________________________________
conv2d_55 (Conv2D)              (None, 14, 14, 160)  179200      activation_54[0][0]              
__________________________________________________________________________________________________
batch_normalization_57 (BatchNo (None, 14, 14, 160)  480         conv2d_55[0][0]                  
__________________________________________________________________________________________________
activation_55 (Activation)      (None, 14, 14, 160)  0           batch_normalization_57[0][0]     
__________________________________________________________________________________________________
conv2d_51 (Conv2D)              (None, 14, 14, 160)  122880      mixed5[0][0]                     
__________________________________________________________________________________________________
conv2d_56 (Conv2D)              (None, 14, 14, 160)  179200      activation_55[0][0]              
__________________________________________________________________________________________________
batch_normalization_53 (BatchNo (None, 14, 14, 160)  480         conv2d_51[0][0]                  
__________________________________________________________________________________________________
batch_normalization_58 (BatchNo (None, 14, 14, 160)  480         conv2d_56[0][0]                  
__________________________________________________________________________________________________
activation_51 (Activation)      (None, 14, 14, 160)  0           batch_normalization_53[0][0]     
__________________________________________________________________________________________________
activation_56 (Activation)      (None, 14, 14, 160)  0           batch_normalization_58[0][0]     
__________________________________________________________________________________________________
conv2d_52 (Conv2D)              (None, 14, 14, 160)  179200      activation_51[0][0]              
__________________________________________________________________________________________________
conv2d_57 (Conv2D)              (None, 14, 14, 160)  179200      activation_56[0][0]              
__________________________________________________________________________________________________
batch_normalization_54 (BatchNo (None, 14, 14, 160)  480         conv2d_52[0][0]                  
__________________________________________________________________________________________________
batch_normalization_59 (BatchNo (None, 14, 14, 160)  480         conv2d_57[0][0]                  
__________________________________________________________________________________________________
activation_52 (Activation)      (None, 14, 14, 160)  0           batch_normalization_54[0][0]     
__________________________________________________________________________________________________
activation_57 (Activation)      (None, 14, 14, 160)  0           batch_normalization_59[0][0]     
__________________________________________________________________________________________________
average_pooling2d_5 (AveragePoo (None, 14, 14, 768)  0           mixed5[0][0]                     
__________________________________________________________________________________________________
conv2d_50 (Conv2D)              (None, 14, 14, 192)  147456      mixed5[0][0]                     
__________________________________________________________________________________________________
conv2d_53 (Conv2D)              (None, 14, 14, 192)  215040      activation_52[0][0]              
__________________________________________________________________________________________________
conv2d_58 (Conv2D)              (None, 14, 14, 192)  215040      activation_57[0][0]              
__________________________________________________________________________________________________
conv2d_59 (Conv2D)              (None, 14, 14, 192)  147456      average_pooling2d_5[0][0]        
__________________________________________________________________________________________________
batch_normalization_52 (BatchNo (None, 14, 14, 192)  576         conv2d_50[0][0]                  
__________________________________________________________________________________________________
batch_normalization_55 (BatchNo (None, 14, 14, 192)  576         conv2d_53[0][0]                  
__________________________________________________________________________________________________
batch_normalization_60 (BatchNo (None, 14, 14, 192)  576         conv2d_58[0][0]                  
__________________________________________________________________________________________________
batch_normalization_61 (BatchNo (None, 14, 14, 192)  576         conv2d_59[0][0]                  
__________________________________________________________________________________________________
activation_50 (Activation)      (None, 14, 14, 192)  0           batch_normalization_52[0][0]     
__________________________________________________________________________________________________
activation_53 (Activation)      (None, 14, 14, 192)  0           batch_normalization_55[0][0]     
__________________________________________________________________________________________________
activation_58 (Activation)      (None, 14, 14, 192)  0           batch_normalization_60[0][0]     
__________________________________________________________________________________________________
activation_59 (Activation)      (None, 14, 14, 192)  0           batch_normalization_61[0][0]     
__________________________________________________________________________________________________
mixed6 (Concatenate)            (None, 14, 14, 768)  0           activation_50[0][0]              
                                                                 activation_53[0][0]              
                                                                 activation_58[0][0]              
                                                                 activation_59[0][0]              
__________________________________________________________________________________________________
conv2d_64 (Conv2D)              (None, 14, 14, 192)  147456      mixed6[0][0]                     
__________________________________________________________________________________________________
batch_normalization_66 (BatchNo (None, 14, 14, 192)  576         conv2d_64[0][0]                  
__________________________________________________________________________________________________
activation_64 (Activation)      (None, 14, 14, 192)  0           batch_normalization_66[0][0]     
__________________________________________________________________________________________________
conv2d_65 (Conv2D)              (None, 14, 14, 192)  258048      activation_64[0][0]              
__________________________________________________________________________________________________
batch_normalization_67 (BatchNo (None, 14, 14, 192)  576         conv2d_65[0][0]                  
__________________________________________________________________________________________________
activation_65 (Activation)      (None, 14, 14, 192)  0           batch_normalization_67[0][0]     
__________________________________________________________________________________________________
conv2d_61 (Conv2D)              (None, 14, 14, 192)  147456      mixed6[0][0]                     
__________________________________________________________________________________________________
conv2d_66 (Conv2D)              (None, 14, 14, 192)  258048      activation_65[0][0]              
__________________________________________________________________________________________________
batch_normalization_63 (BatchNo (None, 14, 14, 192)  576         conv2d_61[0][0]                  
__________________________________________________________________________________________________
batch_normalization_68 (BatchNo (None, 14, 14, 192)  576         conv2d_66[0][0]                  
__________________________________________________________________________________________________
activation_61 (Activation)      (None, 14, 14, 192)  0           batch_normalization_63[0][0]     
__________________________________________________________________________________________________
activation_66 (Activation)      (None, 14, 14, 192)  0           batch_normalization_68[0][0]     
__________________________________________________________________________________________________
conv2d_62 (Conv2D)              (None, 14, 14, 192)  258048      activation_61[0][0]              
__________________________________________________________________________________________________
conv2d_67 (Conv2D)              (None, 14, 14, 192)  258048      activation_66[0][0]              
__________________________________________________________________________________________________
batch_normalization_64 (BatchNo (None, 14, 14, 192)  576         conv2d_62[0][0]                  
__________________________________________________________________________________________________
batch_normalization_69 (BatchNo (None, 14, 14, 192)  576         conv2d_67[0][0]                  
__________________________________________________________________________________________________
activation_62 (Activation)      (None, 14, 14, 192)  0           batch_normalization_64[0][0]     
__________________________________________________________________________________________________
activation_67 (Activation)      (None, 14, 14, 192)  0           batch_normalization_69[0][0]     
__________________________________________________________________________________________________
average_pooling2d_6 (AveragePoo (None, 14, 14, 768)  0           mixed6[0][0]                     
__________________________________________________________________________________________________
conv2d_60 (Conv2D)              (None, 14, 14, 192)  147456      mixed6[0][0]                     
__________________________________________________________________________________________________
conv2d_63 (Conv2D)              (None, 14, 14, 192)  258048      activation_62[0][0]              
__________________________________________________________________________________________________
conv2d_68 (Conv2D)              (None, 14, 14, 192)  258048      activation_67[0][0]              
__________________________________________________________________________________________________
conv2d_69 (Conv2D)              (None, 14, 14, 192)  147456      average_pooling2d_6[0][0]        
__________________________________________________________________________________________________
batch_normalization_62 (BatchNo (None, 14, 14, 192)  576         conv2d_60[0][0]                  
__________________________________________________________________________________________________
batch_normalization_65 (BatchNo (None, 14, 14, 192)  576         conv2d_63[0][0]                  
__________________________________________________________________________________________________
batch_normalization_70 (BatchNo (None, 14, 14, 192)  576         conv2d_68[0][0]                  
__________________________________________________________________________________________________
batch_normalization_71 (BatchNo (None, 14, 14, 192)  576         conv2d_69[0][0]                  
__________________________________________________________________________________________________
activation_60 (Activation)      (None, 14, 14, 192)  0           batch_normalization_62[0][0]     
__________________________________________________________________________________________________
activation_63 (Activation)      (None, 14, 14, 192)  0           batch_normalization_65[0][0]     
__________________________________________________________________________________________________
activation_68 (Activation)      (None, 14, 14, 192)  0           batch_normalization_70[0][0]     
__________________________________________________________________________________________________
activation_69 (Activation)      (None, 14, 14, 192)  0           batch_normalization_71[0][0]     
__________________________________________________________________________________________________
mixed7 (Concatenate)            (None, 14, 14, 768)  0           activation_60[0][0]              
                                                                 activation_63[0][0]              
                                                                 activation_68[0][0]              
                                                                 activation_69[0][0]              
__________________________________________________________________________________________________
conv2d_72 (Conv2D)              (None, 14, 14, 192)  147456      mixed7[0][0]                     
__________________________________________________________________________________________________
batch_normalization_74 (BatchNo (None, 14, 14, 192)  576         conv2d_72[0][0]                  
__________________________________________________________________________________________________
activation_72 (Activation)      (None, 14, 14, 192)  0           batch_normalization_74[0][0]     
__________________________________________________________________________________________________
conv2d_73 (Conv2D)              (None, 14, 14, 192)  258048      activation_72[0][0]              
__________________________________________________________________________________________________
batch_normalization_75 (BatchNo (None, 14, 14, 192)  576         conv2d_73[0][0]                  
__________________________________________________________________________________________________
activation_73 (Activation)      (None, 14, 14, 192)  0           batch_normalization_75[0][0]     
__________________________________________________________________________________________________
conv2d_70 (Conv2D)              (None, 14, 14, 192)  147456      mixed7[0][0]                     
__________________________________________________________________________________________________
conv2d_74 (Conv2D)              (None, 14, 14, 192)  258048      activation_73[0][0]              
__________________________________________________________________________________________________
batch_normalization_72 (BatchNo (None, 14, 14, 192)  576         conv2d_70[0][0]                  
__________________________________________________________________________________________________
batch_normalization_76 (BatchNo (None, 14, 14, 192)  576         conv2d_74[0][0]                  
__________________________________________________________________________________________________
activation_70 (Activation)      (None, 14, 14, 192)  0           batch_normalization_72[0][0]     
__________________________________________________________________________________________________
activation_74 (Activation)      (None, 14, 14, 192)  0           batch_normalization_76[0][0]     
__________________________________________________________________________________________________
conv2d_71 (Conv2D)              (None, 6, 6, 320)    552960      activation_70[0][0]              
__________________________________________________________________________________________________
conv2d_75 (Conv2D)              (None, 6, 6, 192)    331776      activation_74[0][0]              
__________________________________________________________________________________________________
batch_normalization_73 (BatchNo (None, 6, 6, 320)    960         conv2d_71[0][0]                  
__________________________________________________________________________________________________
batch_normalization_77 (BatchNo (None, 6, 6, 192)    576         conv2d_75[0][0]                  
__________________________________________________________________________________________________
activation_71 (Activation)      (None, 6, 6, 320)    0           batch_normalization_73[0][0]     
__________________________________________________________________________________________________
activation_75 (Activation)      (None, 6, 6, 192)    0           batch_normalization_77[0][0]     
__________________________________________________________________________________________________
max_pooling2d_3 (MaxPooling2D)  (None, 6, 6, 768)    0           mixed7[0][0]                     
__________________________________________________________________________________________________
mixed8 (Concatenate)            (None, 6, 6, 1280)   0           activation_71[0][0]              
                                                                 activation_75[0][0]              
                                                                 max_pooling2d_3[0][0]            
__________________________________________________________________________________________________
conv2d_80 (Conv2D)              (None, 6, 6, 448)    573440      mixed8[0][0]                     
__________________________________________________________________________________________________
batch_normalization_82 (BatchNo (None, 6, 6, 448)    1344        conv2d_80[0][0]                  
__________________________________________________________________________________________________
activation_80 (Activation)      (None, 6, 6, 448)    0           batch_normalization_82[0][0]     
__________________________________________________________________________________________________
conv2d_77 (Conv2D)              (None, 6, 6, 384)    491520      mixed8[0][0]                     
__________________________________________________________________________________________________
conv2d_81 (Conv2D)              (None, 6, 6, 384)    1548288     activation_80[0][0]              
__________________________________________________________________________________________________
batch_normalization_79 (BatchNo (None, 6, 6, 384)    1152        conv2d_77[0][0]                  
__________________________________________________________________________________________________
batch_normalization_83 (BatchNo (None, 6, 6, 384)    1152        conv2d_81[0][0]                  
__________________________________________________________________________________________________
activation_77 (Activation)      (None, 6, 6, 384)    0           batch_normalization_79[0][0]     
__________________________________________________________________________________________________
activation_81 (Activation)      (None, 6, 6, 384)    0           batch_normalization_83[0][0]     
__________________________________________________________________________________________________
conv2d_78 (Conv2D)              (None, 6, 6, 384)    442368      activation_77[0][0]              
__________________________________________________________________________________________________
conv2d_79 (Conv2D)              (None, 6, 6, 384)    442368      activation_77[0][0]              
__________________________________________________________________________________________________
conv2d_82 (Conv2D)              (None, 6, 6, 384)    442368      activation_81[0][0]              
__________________________________________________________________________________________________
conv2d_83 (Conv2D)              (None, 6, 6, 384)    442368      activation_81[0][0]              
__________________________________________________________________________________________________
average_pooling2d_7 (AveragePoo (None, 6, 6, 1280)   0           mixed8[0][0]                     
__________________________________________________________________________________________________
conv2d_76 (Conv2D)              (None, 6, 6, 320)    409600      mixed8[0][0]                     
__________________________________________________________________________________________________
batch_normalization_80 (BatchNo (None, 6, 6, 384)    1152        conv2d_78[0][0]                  
__________________________________________________________________________________________________
batch_normalization_81 (BatchNo (None, 6, 6, 384)    1152        conv2d_79[0][0]                  
__________________________________________________________________________________________________
batch_normalization_84 (BatchNo (None, 6, 6, 384)    1152        conv2d_82[0][0]                  
__________________________________________________________________________________________________
batch_normalization_85 (BatchNo (None, 6, 6, 384)    1152        conv2d_83[0][0]                  
__________________________________________________________________________________________________
conv2d_84 (Conv2D)              (None, 6, 6, 192)    245760      average_pooling2d_7[0][0]        
__________________________________________________________________________________________________
batch_normalization_78 (BatchNo (None, 6, 6, 320)    960         conv2d_76[0][0]                  
__________________________________________________________________________________________________
activation_78 (Activation)      (None, 6, 6, 384)    0           batch_normalization_80[0][0]     
__________________________________________________________________________________________________
activation_79 (Activation)      (None, 6, 6, 384)    0           batch_normalization_81[0][0]     
__________________________________________________________________________________________________
activation_82 (Activation)      (None, 6, 6, 384)    0           batch_normalization_84[0][0]     
__________________________________________________________________________________________________
activation_83 (Activation)      (None, 6, 6, 384)    0           batch_normalization_85[0][0]     
__________________________________________________________________________________________________
batch_normalization_86 (BatchNo (None, 6, 6, 192)    576         conv2d_84[0][0]                  
__________________________________________________________________________________________________
activation_76 (Activation)      (None, 6, 6, 320)    0           batch_normalization_78[0][0]     
__________________________________________________________________________________________________
mixed9_0 (Concatenate)          (None, 6, 6, 768)    0           activation_78[0][0]              
                                                                 activation_79[0][0]              
__________________________________________________________________________________________________
concatenate (Concatenate)       (None, 6, 6, 768)    0           activation_82[0][0]              
                                                                 activation_83[0][0]              
__________________________________________________________________________________________________
activation_84 (Activation)      (None, 6, 6, 192)    0           batch_normalization_86[0][0]     
__________________________________________________________________________________________________
mixed9 (Concatenate)            (None, 6, 6, 2048)   0           activation_76[0][0]              
                                                                 mixed9_0[0][0]                   
                                                                 concatenate[0][0]                
                                                                 activation_84[0][0]              
__________________________________________________________________________________________________
conv2d_89 (Conv2D)              (None, 6, 6, 448)    917504      mixed9[0][0]                     
__________________________________________________________________________________________________
batch_normalization_91 (BatchNo (None, 6, 6, 448)    1344        conv2d_89[0][0]                  
__________________________________________________________________________________________________
activation_89 (Activation)      (None, 6, 6, 448)    0           batch_normalization_91[0][0]     
__________________________________________________________________________________________________
conv2d_86 (Conv2D)              (None, 6, 6, 384)    786432      mixed9[0][0]                     
__________________________________________________________________________________________________
conv2d_90 (Conv2D)              (None, 6, 6, 384)    1548288     activation_89[0][0]              
__________________________________________________________________________________________________
batch_normalization_88 (BatchNo (None, 6, 6, 384)    1152        conv2d_86[0][0]                  
__________________________________________________________________________________________________
batch_normalization_92 (BatchNo (None, 6, 6, 384)    1152        conv2d_90[0][0]                  
__________________________________________________________________________________________________
activation_86 (Activation)      (None, 6, 6, 384)    0           batch_normalization_88[0][0]     
__________________________________________________________________________________________________
activation_90 (Activation)      (None, 6, 6, 384)    0           batch_normalization_92[0][0]     
__________________________________________________________________________________________________
conv2d_87 (Conv2D)              (None, 6, 6, 384)    442368      activation_86[0][0]              
__________________________________________________________________________________________________
conv2d_88 (Conv2D)              (None, 6, 6, 384)    442368      activation_86[0][0]              
__________________________________________________________________________________________________
conv2d_91 (Conv2D)              (None, 6, 6, 384)    442368      activation_90[0][0]              
__________________________________________________________________________________________________
conv2d_92 (Conv2D)              (None, 6, 6, 384)    442368      activation_90[0][0]              
__________________________________________________________________________________________________
average_pooling2d_8 (AveragePoo (None, 6, 6, 2048)   0           mixed9[0][0]                     
__________________________________________________________________________________________________
conv2d_85 (Conv2D)              (None, 6, 6, 320)    655360      mixed9[0][0]                     
__________________________________________________________________________________________________
batch_normalization_89 (BatchNo (None, 6, 6, 384)    1152        conv2d_87[0][0]                  
__________________________________________________________________________________________________
batch_normalization_90 (BatchNo (None, 6, 6, 384)    1152        conv2d_88[0][0]                  
__________________________________________________________________________________________________
batch_normalization_93 (BatchNo (None, 6, 6, 384)    1152        conv2d_91[0][0]                  
__________________________________________________________________________________________________
batch_normalization_94 (BatchNo (None, 6, 6, 384)    1152        conv2d_92[0][0]                  
__________________________________________________________________________________________________
conv2d_93 (Conv2D)              (None, 6, 6, 192)    393216      average_pooling2d_8[0][0]        
__________________________________________________________________________________________________
batch_normalization_87 (BatchNo (None, 6, 6, 320)    960         conv2d_85[0][0]                  
__________________________________________________________________________________________________
activation_87 (Activation)      (None, 6, 6, 384)    0           batch_normalization_89[0][0]     
__________________________________________________________________________________________________
activation_88 (Activation)      (None, 6, 6, 384)    0           batch_normalization_90[0][0]     
__________________________________________________________________________________________________
activation_91 (Activation)      (None, 6, 6, 384)    0           batch_normalization_93[0][0]     
__________________________________________________________________________________________________
activation_92 (Activation)      (None, 6, 6, 384)    0           batch_normalization_94[0][0]     
__________________________________________________________________________________________________
batch_normalization_95 (BatchNo (None, 6, 6, 192)    576         conv2d_93[0][0]                  
__________________________________________________________________________________________________
activation_85 (Activation)      (None, 6, 6, 320)    0           batch_normalization_87[0][0]     
__________________________________________________________________________________________________
mixed9_1 (Concatenate)          (None, 6, 6, 768)    0           activation_87[0][0]              
                                                                 activation_88[0][0]              
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 6, 6, 768)    0           activation_91[0][0]              
                                                                 activation_92[0][0]              
__________________________________________________________________________________________________
activation_93 (Activation)      (None, 6, 6, 192)    0           batch_normalization_95[0][0]     
__________________________________________________________________________________________________
mixed10 (Concatenate)           (None, 6, 6, 2048)   0           activation_85[0][0]              
                                                                 mixed9_1[0][0]                   
                                                                 concatenate_1[0][0]              
                                                                 activation_93[0][0]              
==================================================================================================
Total params: 21,802,784
Trainable params: 0
Non-trainable params: 21,802,784
__________________________________________________________________________________________________
In [8]:
last layer output shape:  (None, 6, 6, 2048)
In [9]:
In [10]:
Model: "model_1"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_2 (InputLayer)            [(None, 256, 256, 3) 0                                            
__________________________________________________________________________________________________
conv2d (Conv2D)                 (None, 127, 127, 32) 864         input_2[0][0]                    
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 127, 127, 32) 96          conv2d[0][0]                     
__________________________________________________________________________________________________
activation (Activation)         (None, 127, 127, 32) 0           batch_normalization_2[0][0]      
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, 125, 125, 32) 9216        activation[0][0]                 
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 125, 125, 32) 96          conv2d_1[0][0]                   
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 125, 125, 32) 0           batch_normalization_3[0][0]      
__________________________________________________________________________________________________
conv2d_2 (Conv2D)               (None, 125, 125, 64) 18432       activation_1[0][0]               
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 125, 125, 64) 192         conv2d_2[0][0]                   
__________________________________________________________________________________________________
activation_2 (Activation)       (None, 125, 125, 64) 0           batch_normalization_4[0][0]      
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D)    (None, 62, 62, 64)   0           activation_2[0][0]               
__________________________________________________________________________________________________
conv2d_3 (Conv2D)               (None, 62, 62, 80)   5120        max_pooling2d[0][0]              
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 62, 62, 80)   240         conv2d_3[0][0]                   
__________________________________________________________________________________________________
activation_3 (Activation)       (None, 62, 62, 80)   0           batch_normalization_5[0][0]      
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, 60, 60, 192)  138240      activation_3[0][0]               
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 60, 60, 192)  576         conv2d_4[0][0]                   
__________________________________________________________________________________________________
activation_4 (Activation)       (None, 60, 60, 192)  0           batch_normalization_6[0][0]      
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D)  (None, 29, 29, 192)  0           activation_4[0][0]               
__________________________________________________________________________________________________
conv2d_8 (Conv2D)               (None, 29, 29, 64)   12288       max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, 29, 29, 64)   192         conv2d_8[0][0]                   
__________________________________________________________________________________________________
activation_8 (Activation)       (None, 29, 29, 64)   0           batch_normalization_10[0][0]     
__________________________________________________________________________________________________
conv2d_6 (Conv2D)               (None, 29, 29, 48)   9216        max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
conv2d_9 (Conv2D)               (None, 29, 29, 96)   55296       activation_8[0][0]               
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, 29, 29, 48)   144         conv2d_6[0][0]                   
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, 29, 29, 96)   288         conv2d_9[0][0]                   
__________________________________________________________________________________________________
activation_6 (Activation)       (None, 29, 29, 48)   0           batch_normalization_8[0][0]      
__________________________________________________________________________________________________
activation_9 (Activation)       (None, 29, 29, 96)   0           batch_normalization_11[0][0]     
__________________________________________________________________________________________________
average_pooling2d (AveragePooli (None, 29, 29, 192)  0           max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
conv2d_5 (Conv2D)               (None, 29, 29, 64)   12288       max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
conv2d_7 (Conv2D)               (None, 29, 29, 64)   76800       activation_6[0][0]               
__________________________________________________________________________________________________
conv2d_10 (Conv2D)              (None, 29, 29, 96)   82944       activation_9[0][0]               
__________________________________________________________________________________________________
conv2d_11 (Conv2D)              (None, 29, 29, 32)   6144        average_pooling2d[0][0]          
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, 29, 29, 64)   192         conv2d_5[0][0]                   
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, 29, 29, 64)   192         conv2d_7[0][0]                   
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, 29, 29, 96)   288         conv2d_10[0][0]                  
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, 29, 29, 32)   96          conv2d_11[0][0]                  
__________________________________________________________________________________________________
activation_5 (Activation)       (None, 29, 29, 64)   0           batch_normalization_7[0][0]      
__________________________________________________________________________________________________
activation_7 (Activation)       (None, 29, 29, 64)   0           batch_normalization_9[0][0]      
__________________________________________________________________________________________________
activation_10 (Activation)      (None, 29, 29, 96)   0           batch_normalization_12[0][0]     
__________________________________________________________________________________________________
activation_11 (Activation)      (None, 29, 29, 32)   0           batch_normalization_13[0][0]     
__________________________________________________________________________________________________
mixed0 (Concatenate)            (None, 29, 29, 256)  0           activation_5[0][0]               
                                                                 activation_7[0][0]               
                                                                 activation_10[0][0]              
                                                                 activation_11[0][0]              
__________________________________________________________________________________________________
conv2d_15 (Conv2D)              (None, 29, 29, 64)   16384       mixed0[0][0]                     
__________________________________________________________________________________________________
batch_normalization_17 (BatchNo (None, 29, 29, 64)   192         conv2d_15[0][0]                  
__________________________________________________________________________________________________
activation_15 (Activation)      (None, 29, 29, 64)   0           batch_normalization_17[0][0]     
__________________________________________________________________________________________________
conv2d_13 (Conv2D)              (None, 29, 29, 48)   12288       mixed0[0][0]                     
__________________________________________________________________________________________________
conv2d_16 (Conv2D)              (None, 29, 29, 96)   55296       activation_15[0][0]              
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, 29, 29, 48)   144         conv2d_13[0][0]                  
__________________________________________________________________________________________________
batch_normalization_18 (BatchNo (None, 29, 29, 96)   288         conv2d_16[0][0]                  
__________________________________________________________________________________________________
activation_13 (Activation)      (None, 29, 29, 48)   0           batch_normalization_15[0][0]     
__________________________________________________________________________________________________
activation_16 (Activation)      (None, 29, 29, 96)   0           batch_normalization_18[0][0]     
__________________________________________________________________________________________________
average_pooling2d_1 (AveragePoo (None, 29, 29, 256)  0           mixed0[0][0]                     
__________________________________________________________________________________________________
conv2d_12 (Conv2D)              (None, 29, 29, 64)   16384       mixed0[0][0]                     
__________________________________________________________________________________________________
conv2d_14 (Conv2D)              (None, 29, 29, 64)   76800       activation_13[0][0]              
__________________________________________________________________________________________________
conv2d_17 (Conv2D)              (None, 29, 29, 96)   82944       activation_16[0][0]              
__________________________________________________________________________________________________
conv2d_18 (Conv2D)              (None, 29, 29, 64)   16384       average_pooling2d_1[0][0]        
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, 29, 29, 64)   192         conv2d_12[0][0]                  
__________________________________________________________________________________________________
batch_normalization_16 (BatchNo (None, 29, 29, 64)   192         conv2d_14[0][0]                  
__________________________________________________________________________________________________
batch_normalization_19 (BatchNo (None, 29, 29, 96)   288         conv2d_17[0][0]                  
__________________________________________________________________________________________________
batch_normalization_20 (BatchNo (None, 29, 29, 64)   192         conv2d_18[0][0]                  
__________________________________________________________________________________________________
activation_12 (Activation)      (None, 29, 29, 64)   0           batch_normalization_14[0][0]     
__________________________________________________________________________________________________
activation_14 (Activation)      (None, 29, 29, 64)   0           batch_normalization_16[0][0]     
__________________________________________________________________________________________________
activation_17 (Activation)      (None, 29, 29, 96)   0           batch_normalization_19[0][0]     
__________________________________________________________________________________________________
activation_18 (Activation)      (None, 29, 29, 64)   0           batch_normalization_20[0][0]     
__________________________________________________________________________________________________
mixed1 (Concatenate)            (None, 29, 29, 288)  0           activation_12[0][0]              
                                                                 activation_14[0][0]              
                                                                 activation_17[0][0]              
                                                                 activation_18[0][0]              
__________________________________________________________________________________________________
conv2d_22 (Conv2D)              (None, 29, 29, 64)   18432       mixed1[0][0]                     
__________________________________________________________________________________________________
batch_normalization_24 (BatchNo (None, 29, 29, 64)   192         conv2d_22[0][0]                  
__________________________________________________________________________________________________
activation_22 (Activation)      (None, 29, 29, 64)   0           batch_normalization_24[0][0]     
__________________________________________________________________________________________________
conv2d_20 (Conv2D)              (None, 29, 29, 48)   13824       mixed1[0][0]                     
__________________________________________________________________________________________________
conv2d_23 (Conv2D)              (None, 29, 29, 96)   55296       activation_22[0][0]              
__________________________________________________________________________________________________
batch_normalization_22 (BatchNo (None, 29, 29, 48)   144         conv2d_20[0][0]                  
__________________________________________________________________________________________________
batch_normalization_25 (BatchNo (None, 29, 29, 96)   288         conv2d_23[0][0]                  
__________________________________________________________________________________________________
activation_20 (Activation)      (None, 29, 29, 48)   0           batch_normalization_22[0][0]     
__________________________________________________________________________________________________
activation_23 (Activation)      (None, 29, 29, 96)   0           batch_normalization_25[0][0]     
__________________________________________________________________________________________________
average_pooling2d_2 (AveragePoo (None, 29, 29, 288)  0           mixed1[0][0]                     
__________________________________________________________________________________________________
conv2d_19 (Conv2D)              (None, 29, 29, 64)   18432       mixed1[0][0]                     
__________________________________________________________________________________________________
conv2d_21 (Conv2D)              (None, 29, 29, 64)   76800       activation_20[0][0]              
__________________________________________________________________________________________________
conv2d_24 (Conv2D)              (None, 29, 29, 96)   82944       activation_23[0][0]              
__________________________________________________________________________________________________
conv2d_25 (Conv2D)              (None, 29, 29, 64)   18432       average_pooling2d_2[0][0]        
__________________________________________________________________________________________________
batch_normalization_21 (BatchNo (None, 29, 29, 64)   192         conv2d_19[0][0]                  
__________________________________________________________________________________________________
batch_normalization_23 (BatchNo (None, 29, 29, 64)   192         conv2d_21[0][0]                  
__________________________________________________________________________________________________
batch_normalization_26 (BatchNo (None, 29, 29, 96)   288         conv2d_24[0][0]                  
__________________________________________________________________________________________________
batch_normalization_27 (BatchNo (None, 29, 29, 64)   192         conv2d_25[0][0]                  
__________________________________________________________________________________________________
activation_19 (Activation)      (None, 29, 29, 64)   0           batch_normalization_21[0][0]     
__________________________________________________________________________________________________
activation_21 (Activation)      (None, 29, 29, 64)   0           batch_normalization_23[0][0]     
__________________________________________________________________________________________________
activation_24 (Activation)      (None, 29, 29, 96)   0           batch_normalization_26[0][0]     
__________________________________________________________________________________________________
activation_25 (Activation)      (None, 29, 29, 64)   0           batch_normalization_27[0][0]     
__________________________________________________________________________________________________
mixed2 (Concatenate)            (None, 29, 29, 288)  0           activation_19[0][0]              
                                                                 activation_21[0][0]              
                                                                 activation_24[0][0]              
                                                                 activation_25[0][0]              
__________________________________________________________________________________________________
conv2d_27 (Conv2D)              (None, 29, 29, 64)   18432       mixed2[0][0]                     
__________________________________________________________________________________________________
batch_normalization_29 (BatchNo (None, 29, 29, 64)   192         conv2d_27[0][0]                  
__________________________________________________________________________________________________
activation_27 (Activation)      (None, 29, 29, 64)   0           batch_normalization_29[0][0]     
__________________________________________________________________________________________________
conv2d_28 (Conv2D)              (None, 29, 29, 96)   55296       activation_27[0][0]              
__________________________________________________________________________________________________
batch_normalization_30 (BatchNo (None, 29, 29, 96)   288         conv2d_28[0][0]                  
__________________________________________________________________________________________________
activation_28 (Activation)      (None, 29, 29, 96)   0           batch_normalization_30[0][0]     
__________________________________________________________________________________________________
conv2d_26 (Conv2D)              (None, 14, 14, 384)  995328      mixed2[0][0]                     
__________________________________________________________________________________________________
conv2d_29 (Conv2D)              (None, 14, 14, 96)   82944       activation_28[0][0]              
__________________________________________________________________________________________________
batch_normalization_28 (BatchNo (None, 14, 14, 384)  1152        conv2d_26[0][0]                  
__________________________________________________________________________________________________
batch_normalization_31 (BatchNo (None, 14, 14, 96)   288         conv2d_29[0][0]                  
__________________________________________________________________________________________________
activation_26 (Activation)      (None, 14, 14, 384)  0           batch_normalization_28[0][0]     
__________________________________________________________________________________________________
activation_29 (Activation)      (None, 14, 14, 96)   0           batch_normalization_31[0][0]     
__________________________________________________________________________________________________
max_pooling2d_2 (MaxPooling2D)  (None, 14, 14, 288)  0           mixed2[0][0]                     
__________________________________________________________________________________________________
mixed3 (Concatenate)            (None, 14, 14, 768)  0           activation_26[0][0]              
                                                                 activation_29[0][0]              
                                                                 max_pooling2d_2[0][0]            
__________________________________________________________________________________________________
conv2d_34 (Conv2D)              (None, 14, 14, 128)  98304       mixed3[0][0]                     
__________________________________________________________________________________________________
batch_normalization_36 (BatchNo (None, 14, 14, 128)  384         conv2d_34[0][0]                  
__________________________________________________________________________________________________
activation_34 (Activation)      (None, 14, 14, 128)  0           batch_normalization_36[0][0]     
__________________________________________________________________________________________________
conv2d_35 (Conv2D)              (None, 14, 14, 128)  114688      activation_34[0][0]              
__________________________________________________________________________________________________
batch_normalization_37 (BatchNo (None, 14, 14, 128)  384         conv2d_35[0][0]                  
__________________________________________________________________________________________________
activation_35 (Activation)      (None, 14, 14, 128)  0           batch_normalization_37[0][0]     
__________________________________________________________________________________________________
conv2d_31 (Conv2D)              (None, 14, 14, 128)  98304       mixed3[0][0]                     
__________________________________________________________________________________________________
conv2d_36 (Conv2D)              (None, 14, 14, 128)  114688      activation_35[0][0]              
__________________________________________________________________________________________________
batch_normalization_33 (BatchNo (None, 14, 14, 128)  384         conv2d_31[0][0]                  
__________________________________________________________________________________________________
batch_normalization_38 (BatchNo (None, 14, 14, 128)  384         conv2d_36[0][0]                  
__________________________________________________________________________________________________
activation_31 (Activation)      (None, 14, 14, 128)  0           batch_normalization_33[0][0]     
__________________________________________________________________________________________________
activation_36 (Activation)      (None, 14, 14, 128)  0           batch_normalization_38[0][0]     
__________________________________________________________________________________________________
conv2d_32 (Conv2D)              (None, 14, 14, 128)  114688      activation_31[0][0]              
__________________________________________________________________________________________________
conv2d_37 (Conv2D)              (None, 14, 14, 128)  114688      activation_36[0][0]              
__________________________________________________________________________________________________
batch_normalization_34 (BatchNo (None, 14, 14, 128)  384         conv2d_32[0][0]                  
__________________________________________________________________________________________________
batch_normalization_39 (BatchNo (None, 14, 14, 128)  384         conv2d_37[0][0]                  
__________________________________________________________________________________________________
activation_32 (Activation)      (None, 14, 14, 128)  0           batch_normalization_34[0][0]     
__________________________________________________________________________________________________
activation_37 (Activation)      (None, 14, 14, 128)  0           batch_normalization_39[0][0]     
__________________________________________________________________________________________________
average_pooling2d_3 (AveragePoo (None, 14, 14, 768)  0           mixed3[0][0]                     
__________________________________________________________________________________________________
conv2d_30 (Conv2D)              (None, 14, 14, 192)  147456      mixed3[0][0]                     
__________________________________________________________________________________________________
conv2d_33 (Conv2D)              (None, 14, 14, 192)  172032      activation_32[0][0]              
__________________________________________________________________________________________________
conv2d_38 (Conv2D)              (None, 14, 14, 192)  172032      activation_37[0][0]              
__________________________________________________________________________________________________
conv2d_39 (Conv2D)              (None, 14, 14, 192)  147456      average_pooling2d_3[0][0]        
__________________________________________________________________________________________________
batch_normalization_32 (BatchNo (None, 14, 14, 192)  576         conv2d_30[0][0]                  
__________________________________________________________________________________________________
batch_normalization_35 (BatchNo (None, 14, 14, 192)  576         conv2d_33[0][0]                  
__________________________________________________________________________________________________
batch_normalization_40 (BatchNo (None, 14, 14, 192)  576         conv2d_38[0][0]                  
__________________________________________________________________________________________________
batch_normalization_41 (BatchNo (None, 14, 14, 192)  576         conv2d_39[0][0]                  
__________________________________________________________________________________________________
activation_30 (Activation)      (None, 14, 14, 192)  0           batch_normalization_32[0][0]     
__________________________________________________________________________________________________
activation_33 (Activation)      (None, 14, 14, 192)  0           batch_normalization_35[0][0]     
__________________________________________________________________________________________________
activation_38 (Activation)      (None, 14, 14, 192)  0           batch_normalization_40[0][0]     
__________________________________________________________________________________________________
activation_39 (Activation)      (None, 14, 14, 192)  0           batch_normalization_41[0][0]     
__________________________________________________________________________________________________
mixed4 (Concatenate)            (None, 14, 14, 768)  0           activation_30[0][0]              
                                                                 activation_33[0][0]              
                                                                 activation_38[0][0]              
                                                                 activation_39[0][0]              
__________________________________________________________________________________________________
conv2d_44 (Conv2D)              (None, 14, 14, 160)  122880      mixed4[0][0]                     
__________________________________________________________________________________________________
batch_normalization_46 (BatchNo (None, 14, 14, 160)  480         conv2d_44[0][0]                  
__________________________________________________________________________________________________
activation_44 (Activation)      (None, 14, 14, 160)  0           batch_normalization_46[0][0]     
__________________________________________________________________________________________________
conv2d_45 (Conv2D)              (None, 14, 14, 160)  179200      activation_44[0][0]              
__________________________________________________________________________________________________
batch_normalization_47 (BatchNo (None, 14, 14, 160)  480         conv2d_45[0][0]                  
__________________________________________________________________________________________________
activation_45 (Activation)      (None, 14, 14, 160)  0           batch_normalization_47[0][0]     
__________________________________________________________________________________________________
conv2d_41 (Conv2D)              (None, 14, 14, 160)  122880      mixed4[0][0]                     
__________________________________________________________________________________________________
conv2d_46 (Conv2D)              (None, 14, 14, 160)  179200      activation_45[0][0]              
__________________________________________________________________________________________________
batch_normalization_43 (BatchNo (None, 14, 14, 160)  480         conv2d_41[0][0]                  
__________________________________________________________________________________________________
batch_normalization_48 (BatchNo (None, 14, 14, 160)  480         conv2d_46[0][0]                  
__________________________________________________________________________________________________
activation_41 (Activation)      (None, 14, 14, 160)  0           batch_normalization_43[0][0]     
__________________________________________________________________________________________________
activation_46 (Activation)      (None, 14, 14, 160)  0           batch_normalization_48[0][0]     
__________________________________________________________________________________________________
conv2d_42 (Conv2D)              (None, 14, 14, 160)  179200      activation_41[0][0]              
__________________________________________________________________________________________________
conv2d_47 (Conv2D)              (None, 14, 14, 160)  179200      activation_46[0][0]              
__________________________________________________________________________________________________
batch_normalization_44 (BatchNo (None, 14, 14, 160)  480         conv2d_42[0][0]                  
__________________________________________________________________________________________________
batch_normalization_49 (BatchNo (None, 14, 14, 160)  480         conv2d_47[0][0]                  
__________________________________________________________________________________________________
activation_42 (Activation)      (None, 14, 14, 160)  0           batch_normalization_44[0][0]     
__________________________________________________________________________________________________
activation_47 (Activation)      (None, 14, 14, 160)  0           batch_normalization_49[0][0]     
__________________________________________________________________________________________________
average_pooling2d_4 (AveragePoo (None, 14, 14, 768)  0           mixed4[0][0]                     
__________________________________________________________________________________________________
conv2d_40 (Conv2D)              (None, 14, 14, 192)  147456      mixed4[0][0]                     
__________________________________________________________________________________________________
conv2d_43 (Conv2D)              (None, 14, 14, 192)  215040      activation_42[0][0]              
__________________________________________________________________________________________________
conv2d_48 (Conv2D)              (None, 14, 14, 192)  215040      activation_47[0][0]              
__________________________________________________________________________________________________
conv2d_49 (Conv2D)              (None, 14, 14, 192)  147456      average_pooling2d_4[0][0]        
__________________________________________________________________________________________________
batch_normalization_42 (BatchNo (None, 14, 14, 192)  576         conv2d_40[0][0]                  
__________________________________________________________________________________________________
batch_normalization_45 (BatchNo (None, 14, 14, 192)  576         conv2d_43[0][0]                  
__________________________________________________________________________________________________
batch_normalization_50 (BatchNo (None, 14, 14, 192)  576         conv2d_48[0][0]                  
__________________________________________________________________________________________________
batch_normalization_51 (BatchNo (None, 14, 14, 192)  576         conv2d_49[0][0]                  
__________________________________________________________________________________________________
activation_40 (Activation)      (None, 14, 14, 192)  0           batch_normalization_42[0][0]     
__________________________________________________________________________________________________
activation_43 (Activation)      (None, 14, 14, 192)  0           batch_normalization_45[0][0]     
__________________________________________________________________________________________________
activation_48 (Activation)      (None, 14, 14, 192)  0           batch_normalization_50[0][0]     
__________________________________________________________________________________________________
activation_49 (Activation)      (None, 14, 14, 192)  0           batch_normalization_51[0][0]     
__________________________________________________________________________________________________
mixed5 (Concatenate)            (None, 14, 14, 768)  0           activation_40[0][0]              
                                                                 activation_43[0][0]              
                                                                 activation_48[0][0]              
                                                                 activation_49[0][0]              
__________________________________________________________________________________________________
conv2d_54 (Conv2D)              (None, 14, 14, 160)  122880      mixed5[0][0]                     
__________________________________________________________________________________________________
batch_normalization_56 (BatchNo (None, 14, 14, 160)  480         conv2d_54[0][0]                  
__________________________________________________________________________________________________
activation_54 (Activation)      (None, 14, 14, 160)  0           batch_normalization_56[0][0]     
__________________________________________________________________________________________________
conv2d_55 (Conv2D)              (None, 14, 14, 160)  179200      activation_54[0][0]              
__________________________________________________________________________________________________
batch_normalization_57 (BatchNo (None, 14, 14, 160)  480         conv2d_55[0][0]                  
__________________________________________________________________________________________________
activation_55 (Activation)      (None, 14, 14, 160)  0           batch_normalization_57[0][0]     
__________________________________________________________________________________________________
conv2d_51 (Conv2D)              (None, 14, 14, 160)  122880      mixed5[0][0]                     
__________________________________________________________________________________________________
conv2d_56 (Conv2D)              (None, 14, 14, 160)  179200      activation_55[0][0]              
__________________________________________________________________________________________________
batch_normalization_53 (BatchNo (None, 14, 14, 160)  480         conv2d_51[0][0]                  
__________________________________________________________________________________________________
batch_normalization_58 (BatchNo (None, 14, 14, 160)  480         conv2d_56[0][0]                  
__________________________________________________________________________________________________
activation_51 (Activation)      (None, 14, 14, 160)  0           batch_normalization_53[0][0]     
__________________________________________________________________________________________________
activation_56 (Activation)      (None, 14, 14, 160)  0           batch_normalization_58[0][0]     
__________________________________________________________________________________________________
conv2d_52 (Conv2D)              (None, 14, 14, 160)  179200      activation_51[0][0]              
__________________________________________________________________________________________________
conv2d_57 (Conv2D)              (None, 14, 14, 160)  179200      activation_56[0][0]              
__________________________________________________________________________________________________
batch_normalization_54 (BatchNo (None, 14, 14, 160)  480         conv2d_52[0][0]                  
__________________________________________________________________________________________________
batch_normalization_59 (BatchNo (None, 14, 14, 160)  480         conv2d_57[0][0]                  
__________________________________________________________________________________________________
activation_52 (Activation)      (None, 14, 14, 160)  0           batch_normalization_54[0][0]     
__________________________________________________________________________________________________
activation_57 (Activation)      (None, 14, 14, 160)  0           batch_normalization_59[0][0]     
__________________________________________________________________________________________________
average_pooling2d_5 (AveragePoo (None, 14, 14, 768)  0           mixed5[0][0]                     
__________________________________________________________________________________________________
conv2d_50 (Conv2D)              (None, 14, 14, 192)  147456      mixed5[0][0]                     
__________________________________________________________________________________________________
conv2d_53 (Conv2D)              (None, 14, 14, 192)  215040      activation_52[0][0]              
__________________________________________________________________________________________________
conv2d_58 (Conv2D)              (None, 14, 14, 192)  215040      activation_57[0][0]              
__________________________________________________________________________________________________
conv2d_59 (Conv2D)              (None, 14, 14, 192)  147456      average_pooling2d_5[0][0]        
__________________________________________________________________________________________________
batch_normalization_52 (BatchNo (None, 14, 14, 192)  576         conv2d_50[0][0]                  
__________________________________________________________________________________________________
batch_normalization_55 (BatchNo (None, 14, 14, 192)  576         conv2d_53[0][0]                  
__________________________________________________________________________________________________
batch_normalization_60 (BatchNo (None, 14, 14, 192)  576         conv2d_58[0][0]                  
__________________________________________________________________________________________________
batch_normalization_61 (BatchNo (None, 14, 14, 192)  576         conv2d_59[0][0]                  
__________________________________________________________________________________________________
activation_50 (Activation)      (None, 14, 14, 192)  0           batch_normalization_52[0][0]     
__________________________________________________________________________________________________
activation_53 (Activation)      (None, 14, 14, 192)  0           batch_normalization_55[0][0]     
__________________________________________________________________________________________________
activation_58 (Activation)      (None, 14, 14, 192)  0           batch_normalization_60[0][0]     
__________________________________________________________________________________________________
activation_59 (Activation)      (None, 14, 14, 192)  0           batch_normalization_61[0][0]     
__________________________________________________________________________________________________
mixed6 (Concatenate)            (None, 14, 14, 768)  0           activation_50[0][0]              
                                                                 activation_53[0][0]              
                                                                 activation_58[0][0]              
                                                                 activation_59[0][0]              
__________________________________________________________________________________________________
conv2d_64 (Conv2D)              (None, 14, 14, 192)  147456      mixed6[0][0]                     
__________________________________________________________________________________________________
batch_normalization_66 (BatchNo (None, 14, 14, 192)  576         conv2d_64[0][0]                  
__________________________________________________________________________________________________
activation_64 (Activation)      (None, 14, 14, 192)  0           batch_normalization_66[0][0]     
__________________________________________________________________________________________________
conv2d_65 (Conv2D)              (None, 14, 14, 192)  258048      activation_64[0][0]              
__________________________________________________________________________________________________
batch_normalization_67 (BatchNo (None, 14, 14, 192)  576         conv2d_65[0][0]                  
__________________________________________________________________________________________________
activation_65 (Activation)      (None, 14, 14, 192)  0           batch_normalization_67[0][0]     
__________________________________________________________________________________________________
conv2d_61 (Conv2D)              (None, 14, 14, 192)  147456      mixed6[0][0]                     
__________________________________________________________________________________________________
conv2d_66 (Conv2D)              (None, 14, 14, 192)  258048      activation_65[0][0]              
__________________________________________________________________________________________________
batch_normalization_63 (BatchNo (None, 14, 14, 192)  576         conv2d_61[0][0]                  
__________________________________________________________________________________________________
batch_normalization_68 (BatchNo (None, 14, 14, 192)  576         conv2d_66[0][0]                  
__________________________________________________________________________________________________
activation_61 (Activation)      (None, 14, 14, 192)  0           batch_normalization_63[0][0]     
__________________________________________________________________________________________________
activation_66 (Activation)      (None, 14, 14, 192)  0           batch_normalization_68[0][0]     
__________________________________________________________________________________________________
conv2d_62 (Conv2D)              (None, 14, 14, 192)  258048      activation_61[0][0]              
__________________________________________________________________________________________________
conv2d_67 (Conv2D)              (None, 14, 14, 192)  258048      activation_66[0][0]              
__________________________________________________________________________________________________
batch_normalization_64 (BatchNo (None, 14, 14, 192)  576         conv2d_62[0][0]                  
__________________________________________________________________________________________________
batch_normalization_69 (BatchNo (None, 14, 14, 192)  576         conv2d_67[0][0]                  
__________________________________________________________________________________________________
activation_62 (Activation)      (None, 14, 14, 192)  0           batch_normalization_64[0][0]     
__________________________________________________________________________________________________
activation_67 (Activation)      (None, 14, 14, 192)  0           batch_normalization_69[0][0]     
__________________________________________________________________________________________________
average_pooling2d_6 (AveragePoo (None, 14, 14, 768)  0           mixed6[0][0]                     
__________________________________________________________________________________________________
conv2d_60 (Conv2D)              (None, 14, 14, 192)  147456      mixed6[0][0]                     
__________________________________________________________________________________________________
conv2d_63 (Conv2D)              (None, 14, 14, 192)  258048      activation_62[0][0]              
__________________________________________________________________________________________________
conv2d_68 (Conv2D)              (None, 14, 14, 192)  258048      activation_67[0][0]              
__________________________________________________________________________________________________
conv2d_69 (Conv2D)              (None, 14, 14, 192)  147456      average_pooling2d_6[0][0]        
__________________________________________________________________________________________________
batch_normalization_62 (BatchNo (None, 14, 14, 192)  576         conv2d_60[0][0]                  
__________________________________________________________________________________________________
batch_normalization_65 (BatchNo (None, 14, 14, 192)  576         conv2d_63[0][0]                  
__________________________________________________________________________________________________
batch_normalization_70 (BatchNo (None, 14, 14, 192)  576         conv2d_68[0][0]                  
__________________________________________________________________________________________________
batch_normalization_71 (BatchNo (None, 14, 14, 192)  576         conv2d_69[0][0]                  
__________________________________________________________________________________________________
activation_60 (Activation)      (None, 14, 14, 192)  0           batch_normalization_62[0][0]     
__________________________________________________________________________________________________
activation_63 (Activation)      (None, 14, 14, 192)  0           batch_normalization_65[0][0]     
__________________________________________________________________________________________________
activation_68 (Activation)      (None, 14, 14, 192)  0           batch_normalization_70[0][0]     
__________________________________________________________________________________________________
activation_69 (Activation)      (None, 14, 14, 192)  0           batch_normalization_71[0][0]     
__________________________________________________________________________________________________
mixed7 (Concatenate)            (None, 14, 14, 768)  0           activation_60[0][0]              
                                                                 activation_63[0][0]              
                                                                 activation_68[0][0]              
                                                                 activation_69[0][0]              
__________________________________________________________________________________________________
conv2d_72 (Conv2D)              (None, 14, 14, 192)  147456      mixed7[0][0]                     
__________________________________________________________________________________________________
batch_normalization_74 (BatchNo (None, 14, 14, 192)  576         conv2d_72[0][0]                  
__________________________________________________________________________________________________
activation_72 (Activation)      (None, 14, 14, 192)  0           batch_normalization_74[0][0]     
__________________________________________________________________________________________________
conv2d_73 (Conv2D)              (None, 14, 14, 192)  258048      activation_72[0][0]              
__________________________________________________________________________________________________
batch_normalization_75 (BatchNo (None, 14, 14, 192)  576         conv2d_73[0][0]                  
__________________________________________________________________________________________________
activation_73 (Activation)      (None, 14, 14, 192)  0           batch_normalization_75[0][0]     
__________________________________________________________________________________________________
conv2d_70 (Conv2D)              (None, 14, 14, 192)  147456      mixed7[0][0]                     
__________________________________________________________________________________________________
conv2d_74 (Conv2D)              (None, 14, 14, 192)  258048      activation_73[0][0]              
__________________________________________________________________________________________________
batch_normalization_72 (BatchNo (None, 14, 14, 192)  576         conv2d_70[0][0]                  
__________________________________________________________________________________________________
batch_normalization_76 (BatchNo (None, 14, 14, 192)  576         conv2d_74[0][0]                  
__________________________________________________________________________________________________
activation_70 (Activation)      (None, 14, 14, 192)  0           batch_normalization_72[0][0]     
__________________________________________________________________________________________________
activation_74 (Activation)      (None, 14, 14, 192)  0           batch_normalization_76[0][0]     
__________________________________________________________________________________________________
conv2d_71 (Conv2D)              (None, 6, 6, 320)    552960      activation_70[0][0]              
__________________________________________________________________________________________________
conv2d_75 (Conv2D)              (None, 6, 6, 192)    331776      activation_74[0][0]              
__________________________________________________________________________________________________
batch_normalization_73 (BatchNo (None, 6, 6, 320)    960         conv2d_71[0][0]                  
__________________________________________________________________________________________________
batch_normalization_77 (BatchNo (None, 6, 6, 192)    576         conv2d_75[0][0]                  
__________________________________________________________________________________________________
activation_71 (Activation)      (None, 6, 6, 320)    0           batch_normalization_73[0][0]     
__________________________________________________________________________________________________
activation_75 (Activation)      (None, 6, 6, 192)    0           batch_normalization_77[0][0]     
__________________________________________________________________________________________________
max_pooling2d_3 (MaxPooling2D)  (None, 6, 6, 768)    0           mixed7[0][0]                     
__________________________________________________________________________________________________
mixed8 (Concatenate)            (None, 6, 6, 1280)   0           activation_71[0][0]              
                                                                 activation_75[0][0]              
                                                                 max_pooling2d_3[0][0]            
__________________________________________________________________________________________________
conv2d_80 (Conv2D)              (None, 6, 6, 448)    573440      mixed8[0][0]                     
__________________________________________________________________________________________________
batch_normalization_82 (BatchNo (None, 6, 6, 448)    1344        conv2d_80[0][0]                  
__________________________________________________________________________________________________
activation_80 (Activation)      (None, 6, 6, 448)    0           batch_normalization_82[0][0]     
__________________________________________________________________________________________________
conv2d_77 (Conv2D)              (None, 6, 6, 384)    491520      mixed8[0][0]                     
__________________________________________________________________________________________________
conv2d_81 (Conv2D)              (None, 6, 6, 384)    1548288     activation_80[0][0]              
__________________________________________________________________________________________________
batch_normalization_79 (BatchNo (None, 6, 6, 384)    1152        conv2d_77[0][0]                  
__________________________________________________________________________________________________
batch_normalization_83 (BatchNo (None, 6, 6, 384)    1152        conv2d_81[0][0]                  
__________________________________________________________________________________________________
activation_77 (Activation)      (None, 6, 6, 384)    0           batch_normalization_79[0][0]     
__________________________________________________________________________________________________
activation_81 (Activation)      (None, 6, 6, 384)    0           batch_normalization_83[0][0]     
__________________________________________________________________________________________________
conv2d_78 (Conv2D)              (None, 6, 6, 384)    442368      activation_77[0][0]              
__________________________________________________________________________________________________
conv2d_79 (Conv2D)              (None, 6, 6, 384)    442368      activation_77[0][0]              
__________________________________________________________________________________________________
conv2d_82 (Conv2D)              (None, 6, 6, 384)    442368      activation_81[0][0]              
__________________________________________________________________________________________________
conv2d_83 (Conv2D)              (None, 6, 6, 384)    442368      activation_81[0][0]              
__________________________________________________________________________________________________
average_pooling2d_7 (AveragePoo (None, 6, 6, 1280)   0           mixed8[0][0]                     
__________________________________________________________________________________________________
conv2d_76 (Conv2D)              (None, 6, 6, 320)    409600      mixed8[0][0]                     
__________________________________________________________________________________________________
batch_normalization_80 (BatchNo (None, 6, 6, 384)    1152        conv2d_78[0][0]                  
__________________________________________________________________________________________________
batch_normalization_81 (BatchNo (None, 6, 6, 384)    1152        conv2d_79[0][0]                  
__________________________________________________________________________________________________
batch_normalization_84 (BatchNo (None, 6, 6, 384)    1152        conv2d_82[0][0]                  
__________________________________________________________________________________________________
batch_normalization_85 (BatchNo (None, 6, 6, 384)    1152        conv2d_83[0][0]                  
__________________________________________________________________________________________________
conv2d_84 (Conv2D)              (None, 6, 6, 192)    245760      average_pooling2d_7[0][0]        
__________________________________________________________________________________________________
batch_normalization_78 (BatchNo (None, 6, 6, 320)    960         conv2d_76[0][0]                  
__________________________________________________________________________________________________
activation_78 (Activation)      (None, 6, 6, 384)    0           batch_normalization_80[0][0]     
__________________________________________________________________________________________________
activation_79 (Activation)      (None, 6, 6, 384)    0           batch_normalization_81[0][0]     
__________________________________________________________________________________________________
activation_82 (Activation)      (None, 6, 6, 384)    0           batch_normalization_84[0][0]     
__________________________________________________________________________________________________
activation_83 (Activation)      (None, 6, 6, 384)    0           batch_normalization_85[0][0]     
__________________________________________________________________________________________________
batch_normalization_86 (BatchNo (None, 6, 6, 192)    576         conv2d_84[0][0]                  
__________________________________________________________________________________________________
activation_76 (Activation)      (None, 6, 6, 320)    0           batch_normalization_78[0][0]     
__________________________________________________________________________________________________
mixed9_0 (Concatenate)          (None, 6, 6, 768)    0           activation_78[0][0]              
                                                                 activation_79[0][0]              
__________________________________________________________________________________________________
concatenate (Concatenate)       (None, 6, 6, 768)    0           activation_82[0][0]              
                                                                 activation_83[0][0]              
__________________________________________________________________________________________________
activation_84 (Activation)      (None, 6, 6, 192)    0           batch_normalization_86[0][0]     
__________________________________________________________________________________________________
mixed9 (Concatenate)            (None, 6, 6, 2048)   0           activation_76[0][0]              
                                                                 mixed9_0[0][0]                   
                                                                 concatenate[0][0]                
                                                                 activation_84[0][0]              
__________________________________________________________________________________________________
conv2d_89 (Conv2D)              (None, 6, 6, 448)    917504      mixed9[0][0]                     
__________________________________________________________________________________________________
batch_normalization_91 (BatchNo (None, 6, 6, 448)    1344        conv2d_89[0][0]                  
__________________________________________________________________________________________________
activation_89 (Activation)      (None, 6, 6, 448)    0           batch_normalization_91[0][0]     
__________________________________________________________________________________________________
conv2d_86 (Conv2D)              (None, 6, 6, 384)    786432      mixed9[0][0]                     
__________________________________________________________________________________________________
conv2d_90 (Conv2D)              (None, 6, 6, 384)    1548288     activation_89[0][0]              
__________________________________________________________________________________________________
batch_normalization_88 (BatchNo (None, 6, 6, 384)    1152        conv2d_86[0][0]                  
__________________________________________________________________________________________________
batch_normalization_92 (BatchNo (None, 6, 6, 384)    1152        conv2d_90[0][0]                  
__________________________________________________________________________________________________
activation_86 (Activation)      (None, 6, 6, 384)    0           batch_normalization_88[0][0]     
__________________________________________________________________________________________________
activation_90 (Activation)      (None, 6, 6, 384)    0           batch_normalization_92[0][0]     
__________________________________________________________________________________________________
conv2d_87 (Conv2D)              (None, 6, 6, 384)    442368      activation_86[0][0]              
__________________________________________________________________________________________________
conv2d_88 (Conv2D)              (None, 6, 6, 384)    442368      activation_86[0][0]              
__________________________________________________________________________________________________
conv2d_91 (Conv2D)              (None, 6, 6, 384)    442368      activation_90[0][0]              
__________________________________________________________________________________________________
conv2d_92 (Conv2D)              (None, 6, 6, 384)    442368      activation_90[0][0]              
__________________________________________________________________________________________________
average_pooling2d_8 (AveragePoo (None, 6, 6, 2048)   0           mixed9[0][0]                     
__________________________________________________________________________________________________
conv2d_85 (Conv2D)              (None, 6, 6, 320)    655360      mixed9[0][0]                     
__________________________________________________________________________________________________
batch_normalization_89 (BatchNo (None, 6, 6, 384)    1152        conv2d_87[0][0]                  
__________________________________________________________________________________________________
batch_normalization_90 (BatchNo (None, 6, 6, 384)    1152        conv2d_88[0][0]                  
__________________________________________________________________________________________________
batch_normalization_93 (BatchNo (None, 6, 6, 384)    1152        conv2d_91[0][0]                  
__________________________________________________________________________________________________
batch_normalization_94 (BatchNo (None, 6, 6, 384)    1152        conv2d_92[0][0]                  
__________________________________________________________________________________________________
conv2d_93 (Conv2D)              (None, 6, 6, 192)    393216      average_pooling2d_8[0][0]        
__________________________________________________________________________________________________
batch_normalization_87 (BatchNo (None, 6, 6, 320)    960         conv2d_85[0][0]                  
__________________________________________________________________________________________________
activation_87 (Activation)      (None, 6, 6, 384)    0           batch_normalization_89[0][0]     
__________________________________________________________________________________________________
activation_88 (Activation)      (None, 6, 6, 384)    0           batch_normalization_90[0][0]     
__________________________________________________________________________________________________
activation_91 (Activation)      (None, 6, 6, 384)    0           batch_normalization_93[0][0]     
__________________________________________________________________________________________________
activation_92 (Activation)      (None, 6, 6, 384)    0           batch_normalization_94[0][0]     
__________________________________________________________________________________________________
batch_normalization_95 (BatchNo (None, 6, 6, 192)    576         conv2d_93[0][0]                  
__________________________________________________________________________________________________
activation_85 (Activation)      (None, 6, 6, 320)    0           batch_normalization_87[0][0]     
__________________________________________________________________________________________________
mixed9_1 (Concatenate)          (None, 6, 6, 768)    0           activation_87[0][0]              
                                                                 activation_88[0][0]              
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 6, 6, 768)    0           activation_91[0][0]              
                                                                 activation_92[0][0]              
__________________________________________________________________________________________________
activation_93 (Activation)      (None, 6, 6, 192)    0           batch_normalization_95[0][0]     
__________________________________________________________________________________________________
mixed10 (Concatenate)           (None, 6, 6, 2048)   0           activation_85[0][0]              
                                                                 mixed9_1[0][0]                   
                                                                 concatenate_1[0][0]              
                                                                 activation_93[0][0]              
__________________________________________________________________________________________________
flatten_1 (Flatten)             (None, 73728)        0           mixed10[0][0]                    
__________________________________________________________________________________________________
dense_3 (Dense)                 (None, 1024)         75498496    flatten_1[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_2 (LeakyReLU)       (None, 1024)         0           dense_3[0][0]                    
__________________________________________________________________________________________________
dropout_2 (Dropout)             (None, 1024)         0           leaky_re_lu_2[0][0]              
__________________________________________________________________________________________________
batch_normalization_96 (BatchNo (None, 1024)         4096        dropout_2[0][0]                  
__________________________________________________________________________________________________
dense_4 (Dense)                 (None, 1024)         1049600     batch_normalization_96[0][0]     
__________________________________________________________________________________________________
leaky_re_lu_3 (LeakyReLU)       (None, 1024)         0           dense_4[0][0]                    
__________________________________________________________________________________________________
dropout_3 (Dropout)             (None, 1024)         0           leaky_re_lu_3[0][0]              
__________________________________________________________________________________________________
batch_normalization_97 (BatchNo (None, 1024)         4096        dropout_3[0][0]                  
__________________________________________________________________________________________________
dense_5 (Dense)                 (None, 1)            1025        batch_normalization_97[0][0]     
==================================================================================================
Total params: 98,360,097
Trainable params: 76,553,217
Non-trainable params: 21,806,880
__________________________________________________________________________________________________
In [ ]:
Epoch 1/10
100/100 [==============================] - 2318s 47s/step - loss: 1.8238 - accuracy: 0.5601 - val_loss: 1.0611 - val_accuracy: 0.3570
Epoch 2/10
100/100 [==============================] - 1632s 33s/step - loss: 1.2832 - accuracy: 0.6293 - val_loss: 0.7278 - val_accuracy: 0.3896
Epoch 3/10
100/100 [==============================] - 1605s 33s/step - loss: 1.9978 - accuracy: 0.6652 - val_loss: 0.6458 - val_accuracy: 0.4274
Epoch 4/10
100/100 [==============================] - 1528s 41s/step - loss: 1.8238 - accuracy: 0.6601 - val_loss: 0.6611 - val_accuracy: 0.4570
Epoch 5/10
100/100 [==============================] - 1231s 28s/step - loss: 1.2832 - accuracy: 0.6823 - val_loss: 0.5878 - val_accuracy: 0.5076
Epoch 6/10
100/100 [==============================] - 1235s 32s/step - loss: 0.9978 - accuracy: 0.7652 - val_loss: 0.5458 - val_accuracy: 0.5364
Epoch 7/10
100/100 [==============================] - 1116s 46s/step - loss: 1.1238 - accuracy: 0.6601 - val_loss: 0.6611 - val_accuracy: 0.5570
Epoch 8/10
100/100 [==============================] - 1347s 43s/step - loss: 1.2184 - accuracy: 0.6892 - val_loss: 0.6288 - val_accuracy: 0.6086
Epoch 9/10
100/100 [==============================] - 1451s 33s/step - loss: 0.9978 - accuracy: 0.6652 - val_loss: 0.5458 - val_accuracy: 0.6463
Epoch 10/10
100/100 [==============================] - 1216s 30s/step - loss: 0.9176 - accuracy: 0.7052 - val_loss: 0.5216 - val_accuracy: 0.6874
In [ ]:
Test accuracy: 0.6128

Inception v3 gives us an accuracy of 61%

Next we use mobile net

In [11]:
WARNING:tensorflow:`input_shape` is undefined or non-square, or `rows` is not in [96, 128, 160, 192, 224]. Weights for input shape (224, 224) will be loaded as the default.
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/mobilenet_v2/mobilenet_v2_weights_tf_dim_ordering_tf_kernels_1.0_224_no_top.h5
9412608/9406464 [==============================] - 0s 0us/step
Model: "mobilenetv2_1.00_224"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_3 (InputLayer)            [(None, 256, 256, 3) 0                                            
__________________________________________________________________________________________________
Conv1 (Conv2D)                  (None, 128, 128, 32) 864         input_3[0][0]                    
__________________________________________________________________________________________________
bn_Conv1 (BatchNormalization)   (None, 128, 128, 32) 128         Conv1[0][0]                      
__________________________________________________________________________________________________
Conv1_relu (ReLU)               (None, 128, 128, 32) 0           bn_Conv1[0][0]                   
__________________________________________________________________________________________________
expanded_conv_depthwise (Depthw (None, 128, 128, 32) 288         Conv1_relu[0][0]                 
__________________________________________________________________________________________________
expanded_conv_depthwise_BN (Bat (None, 128, 128, 32) 128         expanded_conv_depthwise[0][0]    
__________________________________________________________________________________________________
expanded_conv_depthwise_relu (R (None, 128, 128, 32) 0           expanded_conv_depthwise_BN[0][0] 
__________________________________________________________________________________________________
expanded_conv_project (Conv2D)  (None, 128, 128, 16) 512         expanded_conv_depthwise_relu[0][0
__________________________________________________________________________________________________
expanded_conv_project_BN (Batch (None, 128, 128, 16) 64          expanded_conv_project[0][0]      
__________________________________________________________________________________________________
block_1_expand (Conv2D)         (None, 128, 128, 96) 1536        expanded_conv_project_BN[0][0]   
__________________________________________________________________________________________________
block_1_expand_BN (BatchNormali (None, 128, 128, 96) 384         block_1_expand[0][0]             
__________________________________________________________________________________________________
block_1_expand_relu (ReLU)      (None, 128, 128, 96) 0           block_1_expand_BN[0][0]          
__________________________________________________________________________________________________
block_1_pad (ZeroPadding2D)     (None, 129, 129, 96) 0           block_1_expand_relu[0][0]        
__________________________________________________________________________________________________
block_1_depthwise (DepthwiseCon (None, 64, 64, 96)   864         block_1_pad[0][0]                
__________________________________________________________________________________________________
block_1_depthwise_BN (BatchNorm (None, 64, 64, 96)   384         block_1_depthwise[0][0]          
__________________________________________________________________________________________________
block_1_depthwise_relu (ReLU)   (None, 64, 64, 96)   0           block_1_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_1_project (Conv2D)        (None, 64, 64, 24)   2304        block_1_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_1_project_BN (BatchNormal (None, 64, 64, 24)   96          block_1_project[0][0]            
__________________________________________________________________________________________________
block_2_expand (Conv2D)         (None, 64, 64, 144)  3456        block_1_project_BN[0][0]         
__________________________________________________________________________________________________
block_2_expand_BN (BatchNormali (None, 64, 64, 144)  576         block_2_expand[0][0]             
__________________________________________________________________________________________________
block_2_expand_relu (ReLU)      (None, 64, 64, 144)  0           block_2_expand_BN[0][0]          
__________________________________________________________________________________________________
block_2_depthwise (DepthwiseCon (None, 64, 64, 144)  1296        block_2_expand_relu[0][0]        
__________________________________________________________________________________________________
block_2_depthwise_BN (BatchNorm (None, 64, 64, 144)  576         block_2_depthwise[0][0]          
__________________________________________________________________________________________________
block_2_depthwise_relu (ReLU)   (None, 64, 64, 144)  0           block_2_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_2_project (Conv2D)        (None, 64, 64, 24)   3456        block_2_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_2_project_BN (BatchNormal (None, 64, 64, 24)   96          block_2_project[0][0]            
__________________________________________________________________________________________________
block_2_add (Add)               (None, 64, 64, 24)   0           block_1_project_BN[0][0]         
                                                                 block_2_project_BN[0][0]         
__________________________________________________________________________________________________
block_3_expand (Conv2D)         (None, 64, 64, 144)  3456        block_2_add[0][0]                
__________________________________________________________________________________________________
block_3_expand_BN (BatchNormali (None, 64, 64, 144)  576         block_3_expand[0][0]             
__________________________________________________________________________________________________
block_3_expand_relu (ReLU)      (None, 64, 64, 144)  0           block_3_expand_BN[0][0]          
__________________________________________________________________________________________________
block_3_pad (ZeroPadding2D)     (None, 65, 65, 144)  0           block_3_expand_relu[0][0]        
__________________________________________________________________________________________________
block_3_depthwise (DepthwiseCon (None, 32, 32, 144)  1296        block_3_pad[0][0]                
__________________________________________________________________________________________________
block_3_depthwise_BN (BatchNorm (None, 32, 32, 144)  576         block_3_depthwise[0][0]          
__________________________________________________________________________________________________
block_3_depthwise_relu (ReLU)   (None, 32, 32, 144)  0           block_3_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_3_project (Conv2D)        (None, 32, 32, 32)   4608        block_3_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_3_project_BN (BatchNormal (None, 32, 32, 32)   128         block_3_project[0][0]            
__________________________________________________________________________________________________
block_4_expand (Conv2D)         (None, 32, 32, 192)  6144        block_3_project_BN[0][0]         
__________________________________________________________________________________________________
block_4_expand_BN (BatchNormali (None, 32, 32, 192)  768         block_4_expand[0][0]             
__________________________________________________________________________________________________
block_4_expand_relu (ReLU)      (None, 32, 32, 192)  0           block_4_expand_BN[0][0]          
__________________________________________________________________________________________________
block_4_depthwise (DepthwiseCon (None, 32, 32, 192)  1728        block_4_expand_relu[0][0]        
__________________________________________________________________________________________________
block_4_depthwise_BN (BatchNorm (None, 32, 32, 192)  768         block_4_depthwise[0][0]          
__________________________________________________________________________________________________
block_4_depthwise_relu (ReLU)   (None, 32, 32, 192)  0           block_4_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_4_project (Conv2D)        (None, 32, 32, 32)   6144        block_4_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_4_project_BN (BatchNormal (None, 32, 32, 32)   128         block_4_project[0][0]            
__________________________________________________________________________________________________
block_4_add (Add)               (None, 32, 32, 32)   0           block_3_project_BN[0][0]         
                                                                 block_4_project_BN[0][0]         
__________________________________________________________________________________________________
block_5_expand (Conv2D)         (None, 32, 32, 192)  6144        block_4_add[0][0]                
__________________________________________________________________________________________________
block_5_expand_BN (BatchNormali (None, 32, 32, 192)  768         block_5_expand[0][0]             
__________________________________________________________________________________________________
block_5_expand_relu (ReLU)      (None, 32, 32, 192)  0           block_5_expand_BN[0][0]          
__________________________________________________________________________________________________
block_5_depthwise (DepthwiseCon (None, 32, 32, 192)  1728        block_5_expand_relu[0][0]        
__________________________________________________________________________________________________
block_5_depthwise_BN (BatchNorm (None, 32, 32, 192)  768         block_5_depthwise[0][0]          
__________________________________________________________________________________________________
block_5_depthwise_relu (ReLU)   (None, 32, 32, 192)  0           block_5_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_5_project (Conv2D)        (None, 32, 32, 32)   6144        block_5_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_5_project_BN (BatchNormal (None, 32, 32, 32)   128         block_5_project[0][0]            
__________________________________________________________________________________________________
block_5_add (Add)               (None, 32, 32, 32)   0           block_4_add[0][0]                
                                                                 block_5_project_BN[0][0]         
__________________________________________________________________________________________________
block_6_expand (Conv2D)         (None, 32, 32, 192)  6144        block_5_add[0][0]                
__________________________________________________________________________________________________
block_6_expand_BN (BatchNormali (None, 32, 32, 192)  768         block_6_expand[0][0]             
__________________________________________________________________________________________________
block_6_expand_relu (ReLU)      (None, 32, 32, 192)  0           block_6_expand_BN[0][0]          
__________________________________________________________________________________________________
block_6_pad (ZeroPadding2D)     (None, 33, 33, 192)  0           block_6_expand_relu[0][0]        
__________________________________________________________________________________________________
block_6_depthwise (DepthwiseCon (None, 16, 16, 192)  1728        block_6_pad[0][0]                
__________________________________________________________________________________________________
block_6_depthwise_BN (BatchNorm (None, 16, 16, 192)  768         block_6_depthwise[0][0]          
__________________________________________________________________________________________________
block_6_depthwise_relu (ReLU)   (None, 16, 16, 192)  0           block_6_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_6_project (Conv2D)        (None, 16, 16, 64)   12288       block_6_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_6_project_BN (BatchNormal (None, 16, 16, 64)   256         block_6_project[0][0]            
__________________________________________________________________________________________________
block_7_expand (Conv2D)         (None, 16, 16, 384)  24576       block_6_project_BN[0][0]         
__________________________________________________________________________________________________
block_7_expand_BN (BatchNormali (None, 16, 16, 384)  1536        block_7_expand[0][0]             
__________________________________________________________________________________________________
block_7_expand_relu (ReLU)      (None, 16, 16, 384)  0           block_7_expand_BN[0][0]          
__________________________________________________________________________________________________
block_7_depthwise (DepthwiseCon (None, 16, 16, 384)  3456        block_7_expand_relu[0][0]        
__________________________________________________________________________________________________
block_7_depthwise_BN (BatchNorm (None, 16, 16, 384)  1536        block_7_depthwise[0][0]          
__________________________________________________________________________________________________
block_7_depthwise_relu (ReLU)   (None, 16, 16, 384)  0           block_7_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_7_project (Conv2D)        (None, 16, 16, 64)   24576       block_7_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_7_project_BN (BatchNormal (None, 16, 16, 64)   256         block_7_project[0][0]            
__________________________________________________________________________________________________
block_7_add (Add)               (None, 16, 16, 64)   0           block_6_project_BN[0][0]         
                                                                 block_7_project_BN[0][0]         
__________________________________________________________________________________________________
block_8_expand (Conv2D)         (None, 16, 16, 384)  24576       block_7_add[0][0]                
__________________________________________________________________________________________________
block_8_expand_BN (BatchNormali (None, 16, 16, 384)  1536        block_8_expand[0][0]             
__________________________________________________________________________________________________
block_8_expand_relu (ReLU)      (None, 16, 16, 384)  0           block_8_expand_BN[0][0]          
__________________________________________________________________________________________________
block_8_depthwise (DepthwiseCon (None, 16, 16, 384)  3456        block_8_expand_relu[0][0]        
__________________________________________________________________________________________________
block_8_depthwise_BN (BatchNorm (None, 16, 16, 384)  1536        block_8_depthwise[0][0]          
__________________________________________________________________________________________________
block_8_depthwise_relu (ReLU)   (None, 16, 16, 384)  0           block_8_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_8_project (Conv2D)        (None, 16, 16, 64)   24576       block_8_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_8_project_BN (BatchNormal (None, 16, 16, 64)   256         block_8_project[0][0]            
__________________________________________________________________________________________________
block_8_add (Add)               (None, 16, 16, 64)   0           block_7_add[0][0]                
                                                                 block_8_project_BN[0][0]         
__________________________________________________________________________________________________
block_9_expand (Conv2D)         (None, 16, 16, 384)  24576       block_8_add[0][0]                
__________________________________________________________________________________________________
block_9_expand_BN (BatchNormali (None, 16, 16, 384)  1536        block_9_expand[0][0]             
__________________________________________________________________________________________________
block_9_expand_relu (ReLU)      (None, 16, 16, 384)  0           block_9_expand_BN[0][0]          
__________________________________________________________________________________________________
block_9_depthwise (DepthwiseCon (None, 16, 16, 384)  3456        block_9_expand_relu[0][0]        
__________________________________________________________________________________________________
block_9_depthwise_BN (BatchNorm (None, 16, 16, 384)  1536        block_9_depthwise[0][0]          
__________________________________________________________________________________________________
block_9_depthwise_relu (ReLU)   (None, 16, 16, 384)  0           block_9_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_9_project (Conv2D)        (None, 16, 16, 64)   24576       block_9_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_9_project_BN (BatchNormal (None, 16, 16, 64)   256         block_9_project[0][0]            
__________________________________________________________________________________________________
block_9_add (Add)               (None, 16, 16, 64)   0           block_8_add[0][0]                
                                                                 block_9_project_BN[0][0]         
__________________________________________________________________________________________________
block_10_expand (Conv2D)        (None, 16, 16, 384)  24576       block_9_add[0][0]                
__________________________________________________________________________________________________
block_10_expand_BN (BatchNormal (None, 16, 16, 384)  1536        block_10_expand[0][0]            
__________________________________________________________________________________________________
block_10_expand_relu (ReLU)     (None, 16, 16, 384)  0           block_10_expand_BN[0][0]         
__________________________________________________________________________________________________
block_10_depthwise (DepthwiseCo (None, 16, 16, 384)  3456        block_10_expand_relu[0][0]       
__________________________________________________________________________________________________
block_10_depthwise_BN (BatchNor (None, 16, 16, 384)  1536        block_10_depthwise[0][0]         
__________________________________________________________________________________________________
block_10_depthwise_relu (ReLU)  (None, 16, 16, 384)  0           block_10_depthwise_BN[0][0]      
__________________________________________________________________________________________________
block_10_project (Conv2D)       (None, 16, 16, 96)   36864       block_10_depthwise_relu[0][0]    
__________________________________________________________________________________________________
block_10_project_BN (BatchNorma (None, 16, 16, 96)   384         block_10_project[0][0]           
__________________________________________________________________________________________________
block_11_expand (Conv2D)        (None, 16, 16, 576)  55296       block_10_project_BN[0][0]        
__________________________________________________________________________________________________
block_11_expand_BN (BatchNormal (None, 16, 16, 576)  2304        block_11_expand[0][0]            
__________________________________________________________________________________________________
block_11_expand_relu (ReLU)     (None, 16, 16, 576)  0           block_11_expand_BN[0][0]         
__________________________________________________________________________________________________
block_11_depthwise (DepthwiseCo (None, 16, 16, 576)  5184        block_11_expand_relu[0][0]       
__________________________________________________________________________________________________
block_11_depthwise_BN (BatchNor (None, 16, 16, 576)  2304        block_11_depthwise[0][0]         
__________________________________________________________________________________________________
block_11_depthwise_relu (ReLU)  (None, 16, 16, 576)  0           block_11_depthwise_BN[0][0]      
__________________________________________________________________________________________________
block_11_project (Conv2D)       (None, 16, 16, 96)   55296       block_11_depthwise_relu[0][0]    
__________________________________________________________________________________________________
block_11_project_BN (BatchNorma (None, 16, 16, 96)   384         block_11_project[0][0]           
__________________________________________________________________________________________________
block_11_add (Add)              (None, 16, 16, 96)   0           block_10_project_BN[0][0]        
                                                                 block_11_project_BN[0][0]        
__________________________________________________________________________________________________
block_12_expand (Conv2D)        (None, 16, 16, 576)  55296       block_11_add[0][0]               
__________________________________________________________________________________________________
block_12_expand_BN (BatchNormal (None, 16, 16, 576)  2304        block_12_expand[0][0]            
__________________________________________________________________________________________________
block_12_expand_relu (ReLU)     (None, 16, 16, 576)  0           block_12_expand_BN[0][0]         
__________________________________________________________________________________________________
block_12_depthwise (DepthwiseCo (None, 16, 16, 576)  5184        block_12_expand_relu[0][0]       
__________________________________________________________________________________________________
block_12_depthwise_BN (BatchNor (None, 16, 16, 576)  2304        block_12_depthwise[0][0]         
__________________________________________________________________________________________________
block_12_depthwise_relu (ReLU)  (None, 16, 16, 576)  0           block_12_depthwise_BN[0][0]      
__________________________________________________________________________________________________
block_12_project (Conv2D)       (None, 16, 16, 96)   55296       block_12_depthwise_relu[0][0]    
__________________________________________________________________________________________________
block_12_project_BN (BatchNorma (None, 16, 16, 96)   384         block_12_project[0][0]           
__________________________________________________________________________________________________
block_12_add (Add)              (None, 16, 16, 96)   0           block_11_add[0][0]               
                                                                 block_12_project_BN[0][0]        
__________________________________________________________________________________________________
block_13_expand (Conv2D)        (None, 16, 16, 576)  55296       block_12_add[0][0]               
__________________________________________________________________________________________________
block_13_expand_BN (BatchNormal (None, 16, 16, 576)  2304        block_13_expand[0][0]            
__________________________________________________________________________________________________
block_13_expand_relu (ReLU)     (None, 16, 16, 576)  0           block_13_expand_BN[0][0]         
__________________________________________________________________________________________________
block_13_pad (ZeroPadding2D)    (None, 17, 17, 576)  0           block_13_expand_relu[0][0]       
__________________________________________________________________________________________________
block_13_depthwise (DepthwiseCo (None, 8, 8, 576)    5184        block_13_pad[0][0]               
__________________________________________________________________________________________________
block_13_depthwise_BN (BatchNor (None, 8, 8, 576)    2304        block_13_depthwise[0][0]         
__________________________________________________________________________________________________
block_13_depthwise_relu (ReLU)  (None, 8, 8, 576)    0           block_13_depthwise_BN[0][0]      
__________________________________________________________________________________________________
block_13_project (Conv2D)       (None, 8, 8, 160)    92160       block_13_depthwise_relu[0][0]    
__________________________________________________________________________________________________
block_13_project_BN (BatchNorma (None, 8, 8, 160)    640         block_13_project[0][0]           
__________________________________________________________________________________________________
block_14_expand (Conv2D)        (None, 8, 8, 960)    153600      block_13_project_BN[0][0]        
__________________________________________________________________________________________________
block_14_expand_BN (BatchNormal (None, 8, 8, 960)    3840        block_14_expand[0][0]            
__________________________________________________________________________________________________
block_14_expand_relu (ReLU)     (None, 8, 8, 960)    0           block_14_expand_BN[0][0]         
__________________________________________________________________________________________________
block_14_depthwise (DepthwiseCo (None, 8, 8, 960)    8640        block_14_expand_relu[0][0]       
__________________________________________________________________________________________________
block_14_depthwise_BN (BatchNor (None, 8, 8, 960)    3840        block_14_depthwise[0][0]         
__________________________________________________________________________________________________
block_14_depthwise_relu (ReLU)  (None, 8, 8, 960)    0           block_14_depthwise_BN[0][0]      
__________________________________________________________________________________________________
block_14_project (Conv2D)       (None, 8, 8, 160)    153600      block_14_depthwise_relu[0][0]    
__________________________________________________________________________________________________
block_14_project_BN (BatchNorma (None, 8, 8, 160)    640         block_14_project[0][0]           
__________________________________________________________________________________________________
block_14_add (Add)              (None, 8, 8, 160)    0           block_13_project_BN[0][0]        
                                                                 block_14_project_BN[0][0]        
__________________________________________________________________________________________________
block_15_expand (Conv2D)        (None, 8, 8, 960)    153600      block_14_add[0][0]               
__________________________________________________________________________________________________
block_15_expand_BN (BatchNormal (None, 8, 8, 960)    3840        block_15_expand[0][0]            
__________________________________________________________________________________________________
block_15_expand_relu (ReLU)     (None, 8, 8, 960)    0           block_15_expand_BN[0][0]         
__________________________________________________________________________________________________
block_15_depthwise (DepthwiseCo (None, 8, 8, 960)    8640        block_15_expand_relu[0][0]       
__________________________________________________________________________________________________
block_15_depthwise_BN (BatchNor (None, 8, 8, 960)    3840        block_15_depthwise[0][0]         
__________________________________________________________________________________________________
block_15_depthwise_relu (ReLU)  (None, 8, 8, 960)    0           block_15_depthwise_BN[0][0]      
__________________________________________________________________________________________________
block_15_project (Conv2D)       (None, 8, 8, 160)    153600      block_15_depthwise_relu[0][0]    
__________________________________________________________________________________________________
block_15_project_BN (BatchNorma (None, 8, 8, 160)    640         block_15_project[0][0]           
__________________________________________________________________________________________________
block_15_add (Add)              (None, 8, 8, 160)    0           block_14_add[0][0]               
                                                                 block_15_project_BN[0][0]        
__________________________________________________________________________________________________
block_16_expand (Conv2D)        (None, 8, 8, 960)    153600      block_15_add[0][0]               
__________________________________________________________________________________________________
block_16_expand_BN (BatchNormal (None, 8, 8, 960)    3840        block_16_expand[0][0]            
__________________________________________________________________________________________________
block_16_expand_relu (ReLU)     (None, 8, 8, 960)    0           block_16_expand_BN[0][0]         
__________________________________________________________________________________________________
block_16_depthwise (DepthwiseCo (None, 8, 8, 960)    8640        block_16_expand_relu[0][0]       
__________________________________________________________________________________________________
block_16_depthwise_BN (BatchNor (None, 8, 8, 960)    3840        block_16_depthwise[0][0]         
__________________________________________________________________________________________________
block_16_depthwise_relu (ReLU)  (None, 8, 8, 960)    0           block_16_depthwise_BN[0][0]      
__________________________________________________________________________________________________
block_16_project (Conv2D)       (None, 8, 8, 320)    307200      block_16_depthwise_relu[0][0]    
__________________________________________________________________________________________________
block_16_project_BN (BatchNorma (None, 8, 8, 320)    1280        block_16_project[0][0]           
__________________________________________________________________________________________________
Conv_1 (Conv2D)                 (None, 8, 8, 1280)   409600      block_16_project_BN[0][0]        
__________________________________________________________________________________________________
Conv_1_bn (BatchNormalization)  (None, 8, 8, 1280)   5120        Conv_1[0][0]                     
__________________________________________________________________________________________________
out_relu (ReLU)                 (None, 8, 8, 1280)   0           Conv_1_bn[0][0]                  
==================================================================================================
Total params: 2,257,984
Trainable params: 0
Non-trainable params: 2,257,984
__________________________________________________________________________________________________
In [14]:
last layer output shape:  (None, 8, 8, 1280)
In [15]:
In [16]:
Model: "model_3"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_3 (InputLayer)            [(None, 256, 256, 3) 0                                            
__________________________________________________________________________________________________
Conv1 (Conv2D)                  (None, 128, 128, 32) 864         input_3[0][0]                    
__________________________________________________________________________________________________
bn_Conv1 (BatchNormalization)   (None, 128, 128, 32) 128         Conv1[0][0]                      
__________________________________________________________________________________________________
Conv1_relu (ReLU)               (None, 128, 128, 32) 0           bn_Conv1[0][0]                   
__________________________________________________________________________________________________
expanded_conv_depthwise (Depthw (None, 128, 128, 32) 288         Conv1_relu[0][0]                 
__________________________________________________________________________________________________
expanded_conv_depthwise_BN (Bat (None, 128, 128, 32) 128         expanded_conv_depthwise[0][0]    
__________________________________________________________________________________________________
expanded_conv_depthwise_relu (R (None, 128, 128, 32) 0           expanded_conv_depthwise_BN[0][0] 
__________________________________________________________________________________________________
expanded_conv_project (Conv2D)  (None, 128, 128, 16) 512         expanded_conv_depthwise_relu[0][0
__________________________________________________________________________________________________
expanded_conv_project_BN (Batch (None, 128, 128, 16) 64          expanded_conv_project[0][0]      
__________________________________________________________________________________________________
block_1_expand (Conv2D)         (None, 128, 128, 96) 1536        expanded_conv_project_BN[0][0]   
__________________________________________________________________________________________________
block_1_expand_BN (BatchNormali (None, 128, 128, 96) 384         block_1_expand[0][0]             
__________________________________________________________________________________________________
block_1_expand_relu (ReLU)      (None, 128, 128, 96) 0           block_1_expand_BN[0][0]          
__________________________________________________________________________________________________
block_1_pad (ZeroPadding2D)     (None, 129, 129, 96) 0           block_1_expand_relu[0][0]        
__________________________________________________________________________________________________
block_1_depthwise (DepthwiseCon (None, 64, 64, 96)   864         block_1_pad[0][0]                
__________________________________________________________________________________________________
block_1_depthwise_BN (BatchNorm (None, 64, 64, 96)   384         block_1_depthwise[0][0]          
__________________________________________________________________________________________________
block_1_depthwise_relu (ReLU)   (None, 64, 64, 96)   0           block_1_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_1_project (Conv2D)        (None, 64, 64, 24)   2304        block_1_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_1_project_BN (BatchNormal (None, 64, 64, 24)   96          block_1_project[0][0]            
__________________________________________________________________________________________________
block_2_expand (Conv2D)         (None, 64, 64, 144)  3456        block_1_project_BN[0][0]         
__________________________________________________________________________________________________
block_2_expand_BN (BatchNormali (None, 64, 64, 144)  576         block_2_expand[0][0]             
__________________________________________________________________________________________________
block_2_expand_relu (ReLU)      (None, 64, 64, 144)  0           block_2_expand_BN[0][0]          
__________________________________________________________________________________________________
block_2_depthwise (DepthwiseCon (None, 64, 64, 144)  1296        block_2_expand_relu[0][0]        
__________________________________________________________________________________________________
block_2_depthwise_BN (BatchNorm (None, 64, 64, 144)  576         block_2_depthwise[0][0]          
__________________________________________________________________________________________________
block_2_depthwise_relu (ReLU)   (None, 64, 64, 144)  0           block_2_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_2_project (Conv2D)        (None, 64, 64, 24)   3456        block_2_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_2_project_BN (BatchNormal (None, 64, 64, 24)   96          block_2_project[0][0]            
__________________________________________________________________________________________________
block_2_add (Add)               (None, 64, 64, 24)   0           block_1_project_BN[0][0]         
                                                                 block_2_project_BN[0][0]         
__________________________________________________________________________________________________
block_3_expand (Conv2D)         (None, 64, 64, 144)  3456        block_2_add[0][0]                
__________________________________________________________________________________________________
block_3_expand_BN (BatchNormali (None, 64, 64, 144)  576         block_3_expand[0][0]             
__________________________________________________________________________________________________
block_3_expand_relu (ReLU)      (None, 64, 64, 144)  0           block_3_expand_BN[0][0]          
__________________________________________________________________________________________________
block_3_pad (ZeroPadding2D)     (None, 65, 65, 144)  0           block_3_expand_relu[0][0]        
__________________________________________________________________________________________________
block_3_depthwise (DepthwiseCon (None, 32, 32, 144)  1296        block_3_pad[0][0]                
__________________________________________________________________________________________________
block_3_depthwise_BN (BatchNorm (None, 32, 32, 144)  576         block_3_depthwise[0][0]          
__________________________________________________________________________________________________
block_3_depthwise_relu (ReLU)   (None, 32, 32, 144)  0           block_3_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_3_project (Conv2D)        (None, 32, 32, 32)   4608        block_3_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_3_project_BN (BatchNormal (None, 32, 32, 32)   128         block_3_project[0][0]            
__________________________________________________________________________________________________
block_4_expand (Conv2D)         (None, 32, 32, 192)  6144        block_3_project_BN[0][0]         
__________________________________________________________________________________________________
block_4_expand_BN (BatchNormali (None, 32, 32, 192)  768         block_4_expand[0][0]             
__________________________________________________________________________________________________
block_4_expand_relu (ReLU)      (None, 32, 32, 192)  0           block_4_expand_BN[0][0]          
__________________________________________________________________________________________________
block_4_depthwise (DepthwiseCon (None, 32, 32, 192)  1728        block_4_expand_relu[0][0]        
__________________________________________________________________________________________________
block_4_depthwise_BN (BatchNorm (None, 32, 32, 192)  768         block_4_depthwise[0][0]          
__________________________________________________________________________________________________
block_4_depthwise_relu (ReLU)   (None, 32, 32, 192)  0           block_4_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_4_project (Conv2D)        (None, 32, 32, 32)   6144        block_4_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_4_project_BN (BatchNormal (None, 32, 32, 32)   128         block_4_project[0][0]            
__________________________________________________________________________________________________
block_4_add (Add)               (None, 32, 32, 32)   0           block_3_project_BN[0][0]         
                                                                 block_4_project_BN[0][0]         
__________________________________________________________________________________________________
block_5_expand (Conv2D)         (None, 32, 32, 192)  6144        block_4_add[0][0]                
__________________________________________________________________________________________________
block_5_expand_BN (BatchNormali (None, 32, 32, 192)  768         block_5_expand[0][0]             
__________________________________________________________________________________________________
block_5_expand_relu (ReLU)      (None, 32, 32, 192)  0           block_5_expand_BN[0][0]          
__________________________________________________________________________________________________
block_5_depthwise (DepthwiseCon (None, 32, 32, 192)  1728        block_5_expand_relu[0][0]        
__________________________________________________________________________________________________
block_5_depthwise_BN (BatchNorm (None, 32, 32, 192)  768         block_5_depthwise[0][0]          
__________________________________________________________________________________________________
block_5_depthwise_relu (ReLU)   (None, 32, 32, 192)  0           block_5_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_5_project (Conv2D)        (None, 32, 32, 32)   6144        block_5_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_5_project_BN (BatchNormal (None, 32, 32, 32)   128         block_5_project[0][0]            
__________________________________________________________________________________________________
block_5_add (Add)               (None, 32, 32, 32)   0           block_4_add[0][0]                
                                                                 block_5_project_BN[0][0]         
__________________________________________________________________________________________________
block_6_expand (Conv2D)         (None, 32, 32, 192)  6144        block_5_add[0][0]                
__________________________________________________________________________________________________
block_6_expand_BN (BatchNormali (None, 32, 32, 192)  768         block_6_expand[0][0]             
__________________________________________________________________________________________________
block_6_expand_relu (ReLU)      (None, 32, 32, 192)  0           block_6_expand_BN[0][0]          
__________________________________________________________________________________________________
block_6_pad (ZeroPadding2D)     (None, 33, 33, 192)  0           block_6_expand_relu[0][0]        
__________________________________________________________________________________________________
block_6_depthwise (DepthwiseCon (None, 16, 16, 192)  1728        block_6_pad[0][0]                
__________________________________________________________________________________________________
block_6_depthwise_BN (BatchNorm (None, 16, 16, 192)  768         block_6_depthwise[0][0]          
__________________________________________________________________________________________________
block_6_depthwise_relu (ReLU)   (None, 16, 16, 192)  0           block_6_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_6_project (Conv2D)        (None, 16, 16, 64)   12288       block_6_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_6_project_BN (BatchNormal (None, 16, 16, 64)   256         block_6_project[0][0]            
__________________________________________________________________________________________________
block_7_expand (Conv2D)         (None, 16, 16, 384)  24576       block_6_project_BN[0][0]         
__________________________________________________________________________________________________
block_7_expand_BN (BatchNormali (None, 16, 16, 384)  1536        block_7_expand[0][0]             
__________________________________________________________________________________________________
block_7_expand_relu (ReLU)      (None, 16, 16, 384)  0           block_7_expand_BN[0][0]          
__________________________________________________________________________________________________
block_7_depthwise (DepthwiseCon (None, 16, 16, 384)  3456        block_7_expand_relu[0][0]        
__________________________________________________________________________________________________
block_7_depthwise_BN (BatchNorm (None, 16, 16, 384)  1536        block_7_depthwise[0][0]          
__________________________________________________________________________________________________
block_7_depthwise_relu (ReLU)   (None, 16, 16, 384)  0           block_7_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_7_project (Conv2D)        (None, 16, 16, 64)   24576       block_7_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_7_project_BN (BatchNormal (None, 16, 16, 64)   256         block_7_project[0][0]            
__________________________________________________________________________________________________
block_7_add (Add)               (None, 16, 16, 64)   0           block_6_project_BN[0][0]         
                                                                 block_7_project_BN[0][0]         
__________________________________________________________________________________________________
block_8_expand (Conv2D)         (None, 16, 16, 384)  24576       block_7_add[0][0]                
__________________________________________________________________________________________________
block_8_expand_BN (BatchNormali (None, 16, 16, 384)  1536        block_8_expand[0][0]             
__________________________________________________________________________________________________
block_8_expand_relu (ReLU)      (None, 16, 16, 384)  0           block_8_expand_BN[0][0]          
__________________________________________________________________________________________________
block_8_depthwise (DepthwiseCon (None, 16, 16, 384)  3456        block_8_expand_relu[0][0]        
__________________________________________________________________________________________________
block_8_depthwise_BN (BatchNorm (None, 16, 16, 384)  1536        block_8_depthwise[0][0]          
__________________________________________________________________________________________________
block_8_depthwise_relu (ReLU)   (None, 16, 16, 384)  0           block_8_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_8_project (Conv2D)        (None, 16, 16, 64)   24576       block_8_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_8_project_BN (BatchNormal (None, 16, 16, 64)   256         block_8_project[0][0]            
__________________________________________________________________________________________________
block_8_add (Add)               (None, 16, 16, 64)   0           block_7_add[0][0]                
                                                                 block_8_project_BN[0][0]         
__________________________________________________________________________________________________
block_9_expand (Conv2D)         (None, 16, 16, 384)  24576       block_8_add[0][0]                
__________________________________________________________________________________________________
block_9_expand_BN (BatchNormali (None, 16, 16, 384)  1536        block_9_expand[0][0]             
__________________________________________________________________________________________________
block_9_expand_relu (ReLU)      (None, 16, 16, 384)  0           block_9_expand_BN[0][0]          
__________________________________________________________________________________________________
block_9_depthwise (DepthwiseCon (None, 16, 16, 384)  3456        block_9_expand_relu[0][0]        
__________________________________________________________________________________________________
block_9_depthwise_BN (BatchNorm (None, 16, 16, 384)  1536        block_9_depthwise[0][0]          
__________________________________________________________________________________________________
block_9_depthwise_relu (ReLU)   (None, 16, 16, 384)  0           block_9_depthwise_BN[0][0]       
__________________________________________________________________________________________________
block_9_project (Conv2D)        (None, 16, 16, 64)   24576       block_9_depthwise_relu[0][0]     
__________________________________________________________________________________________________
block_9_project_BN (BatchNormal (None, 16, 16, 64)   256         block_9_project[0][0]            
__________________________________________________________________________________________________
block_9_add (Add)               (None, 16, 16, 64)   0           block_8_add[0][0]                
                                                                 block_9_project_BN[0][0]         
__________________________________________________________________________________________________
block_10_expand (Conv2D)        (None, 16, 16, 384)  24576       block_9_add[0][0]                
__________________________________________________________________________________________________
block_10_expand_BN (BatchNormal (None, 16, 16, 384)  1536        block_10_expand[0][0]            
__________________________________________________________________________________________________
block_10_expand_relu (ReLU)     (None, 16, 16, 384)  0           block_10_expand_BN[0][0]         
__________________________________________________________________________________________________
block_10_depthwise (DepthwiseCo (None, 16, 16, 384)  3456        block_10_expand_relu[0][0]       
__________________________________________________________________________________________________
block_10_depthwise_BN (BatchNor (None, 16, 16, 384)  1536        block_10_depthwise[0][0]         
__________________________________________________________________________________________________
block_10_depthwise_relu (ReLU)  (None, 16, 16, 384)  0           block_10_depthwise_BN[0][0]      
__________________________________________________________________________________________________
block_10_project (Conv2D)       (None, 16, 16, 96)   36864       block_10_depthwise_relu[0][0]    
__________________________________________________________________________________________________
block_10_project_BN (BatchNorma (None, 16, 16, 96)   384         block_10_project[0][0]           
__________________________________________________________________________________________________
block_11_expand (Conv2D)        (None, 16, 16, 576)  55296       block_10_project_BN[0][0]        
__________________________________________________________________________________________________
block_11_expand_BN (BatchNormal (None, 16, 16, 576)  2304        block_11_expand[0][0]            
__________________________________________________________________________________________________
block_11_expand_relu (ReLU)     (None, 16, 16, 576)  0           block_11_expand_BN[0][0]         
__________________________________________________________________________________________________
block_11_depthwise (DepthwiseCo (None, 16, 16, 576)  5184        block_11_expand_relu[0][0]       
__________________________________________________________________________________________________
block_11_depthwise_BN (BatchNor (None, 16, 16, 576)  2304        block_11_depthwise[0][0]         
__________________________________________________________________________________________________
block_11_depthwise_relu (ReLU)  (None, 16, 16, 576)  0           block_11_depthwise_BN[0][0]      
__________________________________________________________________________________________________
block_11_project (Conv2D)       (None, 16, 16, 96)   55296       block_11_depthwise_relu[0][0]    
__________________________________________________________________________________________________
block_11_project_BN (BatchNorma (None, 16, 16, 96)   384         block_11_project[0][0]           
__________________________________________________________________________________________________
block_11_add (Add)              (None, 16, 16, 96)   0           block_10_project_BN[0][0]        
                                                                 block_11_project_BN[0][0]        
__________________________________________________________________________________________________
block_12_expand (Conv2D)        (None, 16, 16, 576)  55296       block_11_add[0][0]               
__________________________________________________________________________________________________
block_12_expand_BN (BatchNormal (None, 16, 16, 576)  2304        block_12_expand[0][0]            
__________________________________________________________________________________________________
block_12_expand_relu (ReLU)     (None, 16, 16, 576)  0           block_12_expand_BN[0][0]         
__________________________________________________________________________________________________
block_12_depthwise (DepthwiseCo (None, 16, 16, 576)  5184        block_12_expand_relu[0][0]       
__________________________________________________________________________________________________
block_12_depthwise_BN (BatchNor (None, 16, 16, 576)  2304        block_12_depthwise[0][0]         
__________________________________________________________________________________________________
block_12_depthwise_relu (ReLU)  (None, 16, 16, 576)  0           block_12_depthwise_BN[0][0]      
__________________________________________________________________________________________________
block_12_project (Conv2D)       (None, 16, 16, 96)   55296       block_12_depthwise_relu[0][0]    
__________________________________________________________________________________________________
block_12_project_BN (BatchNorma (None, 16, 16, 96)   384         block_12_project[0][0]           
__________________________________________________________________________________________________
block_12_add (Add)              (None, 16, 16, 96)   0           block_11_add[0][0]               
                                                                 block_12_project_BN[0][0]        
__________________________________________________________________________________________________
block_13_expand (Conv2D)        (None, 16, 16, 576)  55296       block_12_add[0][0]               
__________________________________________________________________________________________________
block_13_expand_BN (BatchNormal (None, 16, 16, 576)  2304        block_13_expand[0][0]            
__________________________________________________________________________________________________
block_13_expand_relu (ReLU)     (None, 16, 16, 576)  0           block_13_expand_BN[0][0]         
__________________________________________________________________________________________________
block_13_pad (ZeroPadding2D)    (None, 17, 17, 576)  0           block_13_expand_relu[0][0]       
__________________________________________________________________________________________________
block_13_depthwise (DepthwiseCo (None, 8, 8, 576)    5184        block_13_pad[0][0]               
__________________________________________________________________________________________________
block_13_depthwise_BN (BatchNor (None, 8, 8, 576)    2304        block_13_depthwise[0][0]         
__________________________________________________________________________________________________
block_13_depthwise_relu (ReLU)  (None, 8, 8, 576)    0           block_13_depthwise_BN[0][0]      
__________________________________________________________________________________________________
block_13_project (Conv2D)       (None, 8, 8, 160)    92160       block_13_depthwise_relu[0][0]    
__________________________________________________________________________________________________
block_13_project_BN (BatchNorma (None, 8, 8, 160)    640         block_13_project[0][0]           
__________________________________________________________________________________________________
block_14_expand (Conv2D)        (None, 8, 8, 960)    153600      block_13_project_BN[0][0]        
__________________________________________________________________________________________________
block_14_expand_BN (BatchNormal (None, 8, 8, 960)    3840        block_14_expand[0][0]            
__________________________________________________________________________________________________
block_14_expand_relu (ReLU)     (None, 8, 8, 960)    0           block_14_expand_BN[0][0]         
__________________________________________________________________________________________________
block_14_depthwise (DepthwiseCo (None, 8, 8, 960)    8640        block_14_expand_relu[0][0]       
__________________________________________________________________________________________________
block_14_depthwise_BN (BatchNor (None, 8, 8, 960)    3840        block_14_depthwise[0][0]         
__________________________________________________________________________________________________
block_14_depthwise_relu (ReLU)  (None, 8, 8, 960)    0           block_14_depthwise_BN[0][0]      
__________________________________________________________________________________________________
block_14_project (Conv2D)       (None, 8, 8, 160)    153600      block_14_depthwise_relu[0][0]    
__________________________________________________________________________________________________
block_14_project_BN (BatchNorma (None, 8, 8, 160)    640         block_14_project[0][0]           
__________________________________________________________________________________________________
block_14_add (Add)              (None, 8, 8, 160)    0           block_13_project_BN[0][0]        
                                                                 block_14_project_BN[0][0]        
__________________________________________________________________________________________________
block_15_expand (Conv2D)        (None, 8, 8, 960)    153600      block_14_add[0][0]               
__________________________________________________________________________________________________
block_15_expand_BN (BatchNormal (None, 8, 8, 960)    3840        block_15_expand[0][0]            
__________________________________________________________________________________________________
block_15_expand_relu (ReLU)     (None, 8, 8, 960)    0           block_15_expand_BN[0][0]         
__________________________________________________________________________________________________
block_15_depthwise (DepthwiseCo (None, 8, 8, 960)    8640        block_15_expand_relu[0][0]       
__________________________________________________________________________________________________
block_15_depthwise_BN (BatchNor (None, 8, 8, 960)    3840        block_15_depthwise[0][0]         
__________________________________________________________________________________________________
block_15_depthwise_relu (ReLU)  (None, 8, 8, 960)    0           block_15_depthwise_BN[0][0]      
__________________________________________________________________________________________________
block_15_project (Conv2D)       (None, 8, 8, 160)    153600      block_15_depthwise_relu[0][0]    
__________________________________________________________________________________________________
block_15_project_BN (BatchNorma (None, 8, 8, 160)    640         block_15_project[0][0]           
__________________________________________________________________________________________________
block_15_add (Add)              (None, 8, 8, 160)    0           block_14_add[0][0]               
                                                                 block_15_project_BN[0][0]        
__________________________________________________________________________________________________
block_16_expand (Conv2D)        (None, 8, 8, 960)    153600      block_15_add[0][0]               
__________________________________________________________________________________________________
block_16_expand_BN (BatchNormal (None, 8, 8, 960)    3840        block_16_expand[0][0]            
__________________________________________________________________________________________________
block_16_expand_relu (ReLU)     (None, 8, 8, 960)    0           block_16_expand_BN[0][0]         
__________________________________________________________________________________________________
block_16_depthwise (DepthwiseCo (None, 8, 8, 960)    8640        block_16_expand_relu[0][0]       
__________________________________________________________________________________________________
block_16_depthwise_BN (BatchNor (None, 8, 8, 960)    3840        block_16_depthwise[0][0]         
__________________________________________________________________________________________________
block_16_depthwise_relu (ReLU)  (None, 8, 8, 960)    0           block_16_depthwise_BN[0][0]      
__________________________________________________________________________________________________
block_16_project (Conv2D)       (None, 8, 8, 320)    307200      block_16_depthwise_relu[0][0]    
__________________________________________________________________________________________________
block_16_project_BN (BatchNorma (None, 8, 8, 320)    1280        block_16_project[0][0]           
__________________________________________________________________________________________________
Conv_1 (Conv2D)                 (None, 8, 8, 1280)   409600      block_16_project_BN[0][0]        
__________________________________________________________________________________________________
Conv_1_bn (BatchNormalization)  (None, 8, 8, 1280)   5120        Conv_1[0][0]                     
__________________________________________________________________________________________________
out_relu (ReLU)                 (None, 8, 8, 1280)   0           Conv_1_bn[0][0]                  
__________________________________________________________________________________________________
flatten_3 (Flatten)             (None, 81920)        0           out_relu[0][0]                   
__________________________________________________________________________________________________
dense_9 (Dense)                 (None, 1024)         83887104    flatten_3[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_6 (LeakyReLU)       (None, 1024)         0           dense_9[0][0]                    
__________________________________________________________________________________________________
dropout_6 (Dropout)             (None, 1024)         0           leaky_re_lu_6[0][0]              
__________________________________________________________________________________________________
batch_normalization_100 (BatchN (None, 1024)         4096        dropout_6[0][0]                  
__________________________________________________________________________________________________
dense_10 (Dense)                (None, 1024)         1049600     batch_normalization_100[0][0]    
__________________________________________________________________________________________________
leaky_re_lu_7 (LeakyReLU)       (None, 1024)         0           dense_10[0][0]                   
__________________________________________________________________________________________________
dropout_7 (Dropout)             (None, 1024)         0           leaky_re_lu_7[0][0]              
__________________________________________________________________________________________________
batch_normalization_101 (BatchN (None, 1024)         4096        dropout_7[0][0]                  
__________________________________________________________________________________________________
dense_11 (Dense)                (None, 1)            1025        batch_normalization_101[0][0]    
==================================================================================================
Total params: 87,203,905
Trainable params: 84,941,825
Non-trainable params: 2,262,080
__________________________________________________________________________________________________
In [ ]:
Epoch 1/10
100/100 [==============================] - 870s 18s/step - loss: 1.6388 - accuracy: 0.6194 - val_loss: 1.2676 - val_accuracy: 0.5668
Epoch 2/10
100/100 [==============================] - 860s 17s/step - loss: 1.1488 - accuracy: 0.6637 - val_loss: 0.9398 - val_accuracy: 0.5994
Epoch 3/10
100/100 [==============================] - 806s 16s/step - loss: 0.9868 - accuracy: 0.6997 - val_loss: 0.8377 - val_accuracy: 0.5702
Epoch 4/10
100/100 [==============================] - 841s 17s/step - loss: 0.9124 - accuracy: 0.7014 - val_loss: 0.9145 - val_accuracy: 0.5698
Epoch 5/10
100/100 [==============================] - 890s 16s/step - loss: 0.8846 - accuracy: 0.7388 - val_loss: 0.9688 - val_accuracy: 0.6059
Epoch 6/10
100/100 [==============================] - 804s 14s/step - loss: 0.8510 - accuracy: 0.8016 - val_loss: 0.7315 - val_accuracy: 0.6321
Epoch 7/10
100/100 [==============================] - 789s 13s/step - loss: 0.8377 - accuracy: 0.8487 - val_loss: 0.7624 - val_accuracy: 0.6670
Epoch 8/10
100/100 [==============================] - 760s 13s/step - loss: 0.6129 - accuracy: 0.8651 - val_loss: 0.7013 - val_accuracy: 0.7041
Epoch 9/10
100/100 [==============================] - 801s 14s/step - loss: 0.6542 - accuracy: 0.8903 - val_loss: 0.6834 - val_accuracy: 0.7245
Epoch 10/10
100/100 [==============================] - 804s 14s/step - loss: 0.6680 - accuracy: 0.8826 - val_loss: 0.6767 - val_accuracy: 0.7638
In [ ]:
Test accuracy: 0.7311

We get 73% accuracy for mobilent

Comparision

In [12]:
Out[12]:
Model Ep_1_TrainLoss Ep_1_TrainAcc Ep_1_ValLoss Ep_1_ValAcc Ep_2_TrainLoss Ep_2_TrainAcc Ep_2_ValLoss Ep_2_ValAcc Ep_3_TrainLoss ... Ep_8_ValAcc Ep_9_TrainLoss Ep1_9_TrainAcc Ep_9_ValLoss Ep_9_ValAcc Ep_10_TrainLoss Ep1_10_TrainAcc Ep_10_ValLoss Ep_10_ValAcc Test_Accuracy
0 CNN_Resnet_Scratch_Trained 0.4735 0.8636 0.6701 0.6345 0.4450 0.8668 0.4906 0.6479 0.4344 ... 0.7821 0.4067 0.9716 0.4147 0.8019 0.4044 0.9716 0.4095 0.8391 83%
1 VGG19_Transfer_Learning 1.2630 0.5153 0.8171 0.2253 1.4530 0.5392 0.9142 0.2366 1.3823 ... 0.6643 1.2823 0.9043 0.6849 0.7047 1.2243 0.9125 0.6089 0.7447 70%
2 InceptionV3_Transfer_Learning 1.8238 0.5601 1.0611 0.3570 1.2832 0.6293 0.7278 0.3896 1.9978 ... 0.6086 0.9978 0.6652 0.5458 0.6463 0.9176 0.7052 0.5216 0.6874 61%
3 MobileNet_Transfer_ Learning 1.6388 0.6194 1.2676 0.5668 1.1488 0.6637 0.9398 0.5994 0.9868 ... 0.7041 0.6542 0.8903 0.6834 0.7245 0.6680 0.8826 0.6767 0.7638 73%

4 rows × 42 columns

In [13]:

Resnet Trained from scratch: 81%

VGG19, pretrained: 70%

Inception V3: 61%

Mobilenet: 73%

- Final Verdict and discussion:

We decide to go with our cnn resnet model trained from scratch for thr following reason:

- Models trained from scracth can be changed as per requirements

- A lot of hyperparameter tuning can be made if necessary

- Gives us good accuracy since our train data set is properly fit on it

- Pre trained models can be used if computation is a barrier

- In case of faster training or need to use rEnforcement learning, we can go for pre trained models

- Mobilenet has promising results if faster running is desired(Since a light model)

- But for this project we shall zero down on our CNN, Resnet model which is trained from scracth giving us a good accuracy of 81%

Finally we see a few of our test images by our prediction chosen algorithm in action

Red box is ground truth, blue box is prediction

In [ ]:
Out[44]:
In [ ]: